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1  INTRODUCTION 


TEST  EQUIPMENT  CATEGORIES 

Test,  measurement,  and  diagnostic  equipment  (TMDE)  is  defined  as  any  system  or  device  used 
to  evaluate  the  operational  condition  of  a  system  or  equipment  to  identify  or  isolate  (or  both)  any 
actual  or  potential  malfunction.1  Electric/electronic  TMDE  is  further  subdivided  into  manual 
electronic  test  equipment  (ETE),  automatic  test  equipment  (ATE),  test  program  sets  (TPSs),  and 
calibration  equipment.  The  latter  category  consists  of  several  echelons  of  calibration  standards 
(reference  and  transfer  standards)  to  ensure  traceability  of  measurement  to  national  standards. 

The  principal  distinction  affecting  the  management  and  support  of  ETE  and  ATE  in  the 
Department  of  Defense  (DoD)  is  whether  the  test  equipment  is  "general  purpose”  (i.e.,  common)  or 
"special  purpose”  (i.e. .peculiar).  These  terms  are  defined  in  MIL-STD-1309C  as  follows: 

•  General-ourpose  test  equipment.  Test  equipment  which  is  used  for  the  measurement  of  a 
range  of  parameters  common  to  two  or  more  equipments  or  systems  of  basically  different 
designs. 

•  Special-purpose  test  equipment  Equipment  used  for  test,  repair,  and  maintenance  of  a 
specified  system,  subsystem,  or  module,  having  application  to  only  one  or  a  very  limited 
number  of  systems. 

General-purpose  test  equipment  is  normally  classified  under  Federal  Supply  Class  (FSC)  6625, 
"Electrical  and  Electronic  Properties  Measuring-Testing  Instruments,”  whereas  special-purpose  test 
equipment  can  be  found  under  many  different  FSCs.  Table  1-1  lists  those  FSCs  from  the  total  of 
645  classes/standardization  areas  that,  based  on  title,  may  include  test  equipment. 2  Because  the 
above  definitions  leave  some  latitude,  however,  general-purpose  test  equipment,  if  interpreted  as  any 
ETE  supporting  two  or  more  different  weapons  systems,  can  be  found  in  many  FSCs  other  than 

•Military  Standard  (MIL-STD) -1309C,  "Definitions  of  Terms  for  Test,  Measurement  and 
Diagnostic  Equipment”  (Washington,  D  C  :  Naval  Electronic  Systems  Command,  18  November 
1983) 

'■^Department  of  Defense,  Standardization  Directory,  Defense  Standardization  and  Specifica¬ 
tion  Program,  SD-1  (Washington,  DC.  U  S.  Government  Printing  Office,  1  April  1984) 


TABLE  M.  FEDERAL  SUPPLY  CLASSIFICATION  OF  TEST  EQUIPMENT 


FSC 

TITLE 

ASSIGNEE 

ACTIVITY1 

1190 

Specialized  Test  and  Handling  Equipment,  N uclear  Ordnance 

DS 

1398 

Specialized  Ammo  Handling  and  Servicing  Equipment 

AR 

1450 

Guided  Missile  Handling  and  Servicing  Equipment 

99 

1730 

Aircraft  Ground  Servicing  Equipment 

99 

1850 

Space  Vehicle  Handling  and  Servicing  Equipment 

19 

3470 

Machine  Shop  Sets,  Kits,  and  Outfits 

IP 

3590 

Miscellaneous  Service  and  Trade  Equipment 

GL 

4910 

Motor  Vehicle  Maintenance  and  Repair  Shop  Specialized 

Equipment 

AL 

4920 

Aircraft  Maintenance  and  Repair  Specialized  Equipment 

99 

4921 

Torpedo  Maintenance  Repair  and  Checkout  Specialized  Equipment 

OS 

4923 

Depthcharge  and  Underwater  Mine  Maintenance  Repair  Checkout 
Specialized  Equipment 

OS 

4925 

Ammunition  Maintenance,  Repair  and  Checkout  Specialized 
Equipment 

OS 

4927 

Rocket  Maintenance,  Repair  and  Checkout  Specialized  Equipment 

MI 

4931 

Fire  Control  Maintenance  and  Repair  Shop  Specialized  Equipment 

AR 

4933 

Weapons  Maintenance  and  Repair  Shop  Specialized  Equipment 

AR 

4935 

Guided  Missile  Maintenance,  Repair  and  Checkout  Specialized 
Equipment 

99 

4940 

Miscellaneous  Maintenance  and  Repair  Shop  Specialized 

Equipment 

AL 

4960 

Space  Vehicle  Maintenance,  Repair  and  Checkout  Specialized 
Equipment 

19 

5210 

Measuring  Tools,  Craftsmen 

99 

5220 

Inspection  Gages  and  Precision  Layout  Tools 

AR 

5280 

Sets,  Kits,  and  Outfits  of  Measuring  Tools 

GL 

59 (Group) 

Electrical  and  Electronic  Equipment  Components  (23  FSCs) 

various 

6080 

Fiber  Optic  Kits  and  Sets 

ES 

6540 

Opticians  Instruments,  Equipment  and  Supplies 

DP 

66  (Group) 

Instruments  and  Laboratory  Equipment  (19  FSCs) 

various^ 

7035 

Administrative  Data  Processing  Support  Equipment 

02 

7045 

Administrative  Data  Processing  Supplies  and  Support  Equipment 

02 

•Codes  as  identified  in  Standardization  Directory,  ibid.  AL  =  Armament  Munitions  and 
Chemical  Command;  AR=  Armament  Research  and  Development  Command;  DP  =  Defense 
Personnel  Support  Center,  DS— Defense  Nuclear  Agency;  ES  =  Defense  Electronics  Supply  Center; 
GL  =  Natick  Research  and  Development  Center;  IP  =  Defense  Industrial  Plant  Equipment  Center. 
MI  =  Missile  Command;  OS=  Naval  Sea  Systems  Command  02  =  Directorate  of  Computer  Resources, 
HQ  USAF;  19  =  Space  Division  (Air  Force  Systems  Command).  99=  Air  Force  Logistics  Command 
Cataloging  and  Standardization  Center 

^Includes  FSC  6625  for  which  the  Army's  Communications-Klectronics  Command  (CECOM)  is 
the  assignee  activity. 


FSC  6625;  conversely,  special-purpose  test  equipment  may  be  found  in  FSC  6625  In  addition,  a  piece 
of  test  equipment  for  a  new  weapons  system  may  initially  be  classified  as  special  purpose,  but  then 
later  be  reclassified  as  general  purpose  when  it  is  used  in  support  of  other  weapons  systems.  Its 
initial  classification,  however,  may  not  be  changed.  Furthermore,  the  Military  Services  may  use 
different  ancillary  items  (e  g.,  plug-ins)  for  the  same  basic  test  instrument,  so  that  different  National 
Stock  Numbers  (NSNs)  may  apply  to  the  same  basic  instrument  stocked  with  different  ancillary 
items.  Additionally,  such  an  ancillary  item  may  be  peculiar  to  a  particular  weapons  system  so  that 
an  item  of  general-purpose  test  equipment  is  actually  used,  and  may  be  so  classified,  as  special 
purpose. 

Generally,  special-purpose  test  equipment  is  managed  in  conjunction  with  the  supported  prime 
weapons  system  or  subsystem(s),  whereas  general-purpose  test  equipment  is  managed  by  a  separate 
functional  manager.  Importantly,  the  most  recent  edition  of  the  DoD  Standardization  Directory 
specifies  that  ATE  and  its  accessories  be  classified  under  FSC  6625  —  a  new  requirement  that  confuses 
the  distinction  between  general-purpose  and  special-purpose  ATE. 

The  acquisition  and  life  cycle  support  of  TPSs  is  also  generally  managed  in  conjunction  with 
the  associated  prime  equipment  assemblies  or  modules,  while  that  of  calibration  equipment  is  a 
separate  functional  area  within  each  of  the  Military  Services 
DEFENSE  POLICY  GUIDANCE 

The  acquisition  and  life  cycle  support  of  test  equipment  are  management  functions  that  have 
been  delegated  to  the  Military  Services  by  the  Office  of  the  Secretary  of  Defense  (OSD)  Existing  DoD 
directives  and  instructions  pertaining  to  acquisition  focus  primarily  on  prime  equipment,  with  sup¬ 
port  equipment  treated  as  an  integrated  logistic  support  (ILS)  element.  With  respect  to  the  latter,  the 
policy  emphasis  is  on  (Dearly  identification,  tradeoffs,  and  determination  of  ILS  requirements 
through  an  adequately  funded  ILS  program  and  Logistic  Support  Analysis  (LSA)  effort;  and 
(2)  adequacy  of  programmed  support  resources  to  meet  prime  equipment  readiness  requirements, 
considering  initial  and  mature  reliability  and  maintainability  values  and  field  experience  on  similar 


equipment.  DoD  policy  on  the  acquisition  and  support  of  test  equipment  is  described  in  detail  in 
Volume  IV  of  this  report. 

With  respect  to  the  management  and  support  of  materiel  once  it  enters  the  DoD  inventory, 
current  directives  and  instructions,  with  some  exceptions,  do  not  discriminate  between  prime  and 
support  equipment.  Moreover,  the  various  management  aspects  (e  g.,  standardization,  maintenance 
concept/plan,  provisioning,  diminishing  manufacturing  sources,  quality  assurance,  economic  repair 
limits,  technological  obsolescence,  and  monitoring  or  reporting  of  availability)  are  addressed  individ¬ 
ually  and  separately  in  numerous  documents. 

REPORT  ORGANIZATION 

The  Military  Services’  organizations  and  procedures  for  test  equipment  acquisition  and  inven¬ 
tory  management  are  described  in  Chapter  2;  those  for  test  equipment  support  (calibration  and 
repair)  are  presented  in  Chapter  3.  The  acquisition  and  support  procedures  for  TPSs  are  described 
separately  in  Chapter  4. 

Appendix  A  shows  the  Army's  charter  for  its  Program  Manager  for  TMDE  Appendices  B,  C, 
and  D  summarize  the  standard  management  approaches  for  support  equipment  and  ATE  acquisition 
that  have  been  adopted  by  the  Military  Services.  Appendix  E  describes  the  Navy's  unique  approach 
to  support  equipment  selection,  while  Appendix  F  details  the  Navy’s  planned  approach  (not  yet  offi¬ 
cially  approved)  to  including  operating  and  support  costs  for  commercial  test  equipment  in  procure 
ment  contracts  through  the  concept  of  life  cycle  warranties 


2.  TEST  EQUIPMENT  ACQUISITION  AND  INVENTORY  MANAGEMENT 


This  chapter  describes  the  management  structures,  processes,  and  procedures  in  the  Military 
Services  for  test  equipment  acquisition  and  inventory  management.  It  focuses  on  the  similarities  and 
differences  in  the  various  management  functions,  including  acquisition  of  new  test  equipment;  pro¬ 
curement  of  common  ETE;  development  of  standard  ATE;  inventory  management  (planning,  inven¬ 
tory  control,  and  modernization);  and  planning  for  wartime  requirements. 

MANAGEMENT  STRUCTURE 

Within  each  of  the  Military  Services,  management  of  peculiar  test  equipment  is  the  respon¬ 
sibility  of  the  acquisition  or  life  cycle  manager  of  the  supported  weapons  system,  while  the  acquisition 
and  life  cycle  management  of  common  test  equipment  is  the  responsibility  of  a  separate  manager. 
The  primary  differences  occur  in  the  authority  and  control  given  to  the  common  test  equipment 
managers 

The  Army  has  the  tightest,  most  centralized  management  structure,  with  a  single  program 
manager  for  both  acquisition  and  life  cycle  management  of  all  common  test  equipment.  This  program 
manager  reports  to  the  Executive  Director  for  TMDE,  Office  of  the  Deputy  Commanding  General  for 
Materiel  Readiness,  Army  Materiel  Command  (AMC),  and  possesses  the  requisite  authority  and  con¬ 
trol  for  effective  management  of  common  test  equipment  (see  Appendix  A I 

The  Marine  Corps,  on  the  other  hand,  separates  the  acquisition  and  life  cycle  management  of 
common  test  equipment.  The  Support  and  Test  Equipment  Section,  Materiel  Division,  Headquarters 
Marine  Corps,  is  the  centralized  procurement  manager  for  principal  end  items  of  common  test  equip¬ 
ment,  excluding  secondary  items  and  low-cost  end  items;  whereas  the  Marine  Corps  Logistics  Base, 
Albany,  Georgia,  is  the  principal  inventory  or  item  manager  of  common  test  equipment  after  it  enters 
the  inventory  Moreover,  the  procurement  manager  has  little  influence  or  control  over  the  selection 
of  test  equipment  to  he  purchased  for  new  weapons  systems 


The  Navy  and  Air  Force  management  structures  are  between  these  two  extremes.  The  Navy 
has  a  central  manager  for  the  acquisition  and  life  cycle  management  of  common  test  equipment  —  the 
Test  and  Monitoring  Systems  Group  of  the  Life  Cycle  Engineering  and  Platform  Integration 
Directorate,  Naval  Electronic  Systems  Command  (NAVELEX).  Navy-wide  management  responsi¬ 
bility  of  this  group  is  limited  to  what  the  Navy  refers  to  as  "standard  GPETE  [general-purpose  ETE]”; 
acquisition  and  life  cycle  management  of  nonstandard  common  test  equipment,  low-demand  common 
test  equipment,  and  peculiar  test  equipment  are  dispersed  among  various  test  equipment  item 
managers  in  the  Systems  Commands.  The  contracting  for  common  test  equipment  and  day-to-day 
inventory  management  (item  management)  has  been  delegated  by  NAVELEX  to  the  Ships  Parts 
Control  Center  (SPCC),  Mechanicsburg,  Pennsylvania,  a  field  activity  of  the  Naval  Supply  Systems 
Command.  The  Navy’s  management  structure  is  actually  designed  to  foster  standardization  within 
each  Systems  Command,  rather  than  Navy- wide.  Each  of  the  Systems  Commands  has  a  centralized 
test  equipment  manager  responsible  for  test  equipment  requirements  review  and  approval. 

The  Air  Force  has  designated  the  San  Antonio  Air  Logistics  Center  (ALC),  Texas,  as  the  cen¬ 
tral  manager  of  common  test  equipment,  but  its  responsibility  is  limited  to  test  equipment  classified 
as  FSC  6625.  (It  is  also  the  "system  manager”  for  standard  ATE  )  The  acquisition  and  life  cycle  man¬ 
agement  of  other  common  test  equipment  is  dispersed  among  support  equipment  system  program 
offices  in  the  Air  Force  Systems  Command  (AFSC)  and  various  system  and  item  managers  in  the 
ALCs.  A  small  office  at  Air  Force  Logistics  Command  (AFLC)  Headquarters  coordinates  all  test 
equipment  support. 

The  separation  of  management  responsibilities  for  acquisition  decisions  and  life  cycle  manage¬ 
ment  of  common  test  equipment  in  both  the  Navy  and  Air  Force  appear  to  be  a  weak  link  that  has 
been  avoided  in  the  Army’s  centralized  management  structure.  The  effectiveness  of  the  centralized 
manager  of  common  test  equipment  in  both  the  Navy  and  Air  Force  is  further  limited  by  their  lack  of 
influence  on  test  equipment  decisions  made  during  the  weapons  system  acquisition  process  Neither 
has  the  type  of  authority  that  the  Army  has  delegated  to  its  program  manager  for  TMDE. 


ACQUISITION  MANAGEMENT  PROCESS 

The  overall  management  approach  to  the  acquisition  of  test  equipment  is  uniform  among  the 
Military  Services  because  both  the  process  and  procedures  have  been  standardized  by  the  weapons 
system  acquisition  process.  The  LSA  process  is  defined  in  a  military  standard1  and  the  management 
procedures  are  described  in  a  Joint  Service  regulation,  "Standard  Integrated  Support  Management 
System"  (SISMS).  A  summary  of  the  LSA  process  is  provided  in  Appendix  B,  while  an  extract  of 
SISMS  is  included  as  Appendix  C. 

The  current  version  of  MIL-STD-1388  represents  a  major  improvement  over  the  previous  ver¬ 
sion  (dated  15  October  1973)  and  is  normally  invoked  (although  tailored  to  the  particular  circum¬ 
stances)  in  most  acquisition  programs.  Both  the  Air  Force  and  the  Navy  customarily  apply  SISMS  for 
major  acquisition  programs,  but  the  Army  does  so  only  when  it  is  involved  in  a  muiti-Service  acquisi¬ 
tion  program. 

In  addition  to  the  LSA  and  SISMS,  the  Military  Services  use  jointly  developed  guides  for  the 
acquisition  of  automatic  testing  capabilities  (both  built-in-test  and  off-line  ATE)  Those  guides  are 
summarized  in  Appendix  D  A  description  of  the  acquisition  management  process  as  implemented  by 
the  Army,  Navy,  and  Air  Force  follows 

Army 

With  the  1982  reorganization  of  its  management  of  TMDE,  the  Army  adopted  a  central 
ized  system  of  acquisition  approval  and  registration  that  is  conducted  by  the  Central  TMDE  Activity 
(CTA),  Lexington  Bluegrass  Depot,  Kentucky,  under  the  authority  of  the  Commanding  General  AMC 
as  Department  of  the  Army  (DA)  executive  agent  for  TMDE  The  CTA  performs  the  Army-wide 
inventory  management  of  TMDE  used  in  the  maintenance  of  equipment  in  the  fielded  Army,  but  this 
role  is  limited  to  the  category  of  TMDE  supported  by  Army  TMDE  support  units  [i.e..  common  TMDE 


•MIL-STD-ISSS-IA,  "Logistic  Support  Analysis,"  11  April  1983,  and  MIL  STD-1388  2A. 
"DoD  Requirements  for  a  Logistic  Support  Analysis  Record,”  20Julv  1984  (Washington,  D  C 
Department  of  Defense) 


and  selected  items  of  peculiar  TMDE  subject  to  agreement  by  the  U.S.  Army  TMDE  Support  Group 
(USATSG),  Redstone  Arsenall.  Its  functions  in  support  of  this  responsibility  include  the  following: 

•  Conducting  assessments  of  TMDE  to  identify  problems  in  performance,  support- 
ability,  or  schedule. 

•  Maintaining  the  Army  "Preferred  Items  List  (PIL)  for  TMDE”  (DA  Pam¬ 
phlet  700-21-1)  as  well  as  the  "Army  TMDE  Register”  (DA  Pamphlet  700-20),  which 
lists  all  common  TMDE  in  the  Army  inventory  or  approved  for  Army  use. 

•  Publishing  the  "Equipment  Improvement  Report  and  Maintenance  Digest  for 
TMDE”  (Technical  Bulletin  43-000 1-61)  as  a  means  of  disseminating  information 
and  lessons  learned  to  Active  and  Reserve  Component  TMDE  users. 

•  Developing  and  maintaining  the  Army’s  TMDE  management  information  system, 
which  is  intended  to  provide  a  single  source  of  information  on  TMDE  inventory 
position,  maintenance  history,  performance,  and  other  characteristics. 

All  these  functions  fall  under  inventory  management.  One  additional  function  of  the  CTA,  and  the 
primary  reason  why  it  is  mentioned  in  this  section  on  acquisition  management,  is  its  approval 
authority  for  most  TMDE  acquisitions  in  the  Army.  (Some  TMDE,  such  as  mechanical  gages  and 
certain  items  managed  by  the  Army  Intelligence  and  Security  Command,  are  exempt  and  identified 
in  Appendix  B  of  Army  Regulation  750-43,  "Test,  Measurement  and  Diagnostic  Equipment,” 
April  1984.)  The  TMDE  selection  and  acquisition  approval  process  is  described  below. 2 

Test  equipment  requirements  are  first  identified  in  the  LSA  process  Normally,  the  con¬ 
tractor  will  document  LSA  results  in  the  LSA  record  Data  Sheet  C  (Maintenance  Task  Analysis)  and 
then  screen  the  various  test  equipment  inventory  lists  in  preparing  LSA  record  Data  Sheet  E 
("Support  Equipment,  Special  Tools  or  Training  Equipment  Description  and  Justification”).  Con¬ 
tracts  typically  require  the  following  order  of  priority  for  test  equipment  selection:3 

•  DA  Pamphlet  700-21-1 

•  DA  Pamphlet  700-20 

2For  a  more  detailed  description  and  recent  case  examples  of  Army  acquisition  programs,  the 
reader  is  referred  to:  Giordano  Associates,  Inc  ,  LSA  TMDE  Requirements  Analysis  Study  (Corona, 
California:  Fleet  Analysis  Center,  30  September  1983). 

3The  list  has  been  adopted,  with  some  changes,  from  the  TMDE  selection  criteria  stated  in 
the  AN/TPQ-36  Firefinder  radar  contract. 


•  DA  Supply  Catalog 

•  Other  Government/Do D  inventory  items  as  listed  in: 


Military  Handbook  (MIL-HDBK)  -300  [U.S.  Air  Force  (USAF)l,  "Technical 
Information  File  of  Support  Equipment” 

Technical  Order  (T.O.)  33K-2-I00  (Air  Force),  "Precision  Measuring  Equipment” 

T.O.  33-1-1,  33-1-2,  and  33-1-3  (Air  Force),  "Electronic  Test  Equipment  Nomen¬ 
clature  Assignments,  Government-Industry  Data  Exchange  Program  (GIDEP)” 

MIL-STD- 1364  (Navy),  "Standard  General  Purpose  Electronic  Test  Equipment” 

Technical  Manual  (TM)  6625-15/1  (Marine  Corps),  "Electronic  Nomenclature 
Assignments” 

Hewlett-Packard  and  Tektronix  Logistics  Data  Books 

•  Modification  of  existing  military  specifications  test  equipment 

•  Off-the-shelf  commercial  items 

! 

i  •  Modification  of  commercial  items 

l 

i  •  Development  of  new  test  equipment  to  meet  peculiar  requirements. 

I  Depending  on  the  way  the  contract  is  worded  and  the  contract  data  requirements  list, 

\ 

the  contractor-proposed  test  equipment  may  be  documented  in  the  LSA  record  or  in  the  form  of 

/  Ground  Support  Equipment  Recommendation  Data  in  accordance  with  a  standard  data  item  descrip- 

) 

i  tion.  Data  Item  (DI)  -S-6176.  The  ILS  management  team  assigned  to  the  program  reviews  this 

1  documentation  with  final  approval  or  disapproval  by  the  program  manager  based  on  adequacy  of  the 

v 

justification  and  cost  effectiveness  of  the  proposed  approach.  Following  approval  by  the  program 
manager,  the  contractor  prepares  a  functional  description  for  any  new  items  on  the  approved  list  of 
test  equipment. 

The  program  management  office  prepares  a  DA  Form4062-1-R  ("TMDE  Requirements 
I  Review”)  for  each  approved  test  equipment  item  (as  well  as  a  DA  Form4062-R,  "TMDE  Item  Tech 

,  nical  Description,”  for  any  new  item),  and  this  documentation  is  forwarded  with  a  letter  "clearance 

j  request  for  TMDE”  to  the  CTA  for  approval  and  to  the  item  manager  for  information  The  item 

I  manager  for  most  common  ETE  is  located  in  Communications-Electronics  Command  (CECOM)  The 

I 

I 

I 

i  i 

i  i 
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requesting  activity,  i.e.,  program  management  office,  may  be  located  in  any  one  of  the  other  major 
subordinate  commands  of  AMC.  The  item  manager  is  responsible  for  entering  the  item  in  The  Army 
Authorization  Document  System  and  for  type  classification  or  NSN  assignment  The  CTA,  in 
coordination  with  the  Program  Manager  for  TMDE  and  the  L'SATSG,  approves  or  disapproves  the 
selected  test  equipment  items 

In  the  event  of  disapproval  by  the  CTA,  the  requesting  activity  may  submit  additional 
justification  data  for  reevaluation,  with  unresolved  problems  brought  to  the  attention  of  the 
Executive  Director  for  TMDE.  Following  that  approval,  the  re-uesting  activity  or  item  manager 
initiates  procurement  of  the  approved  item  For  any  new  items,  CTA  updates  the  TMDE  register 
with  the  information  from  DA  Form  4062-R. 

The  Army  PIL,  which  is  updated  annually,  currently  contains  137  items  of  common 
ETE.  The  TMDE  Register  is  published  in  two  parts,  an  index  listing  on  paper  and  a  Technical 
Description  section  on  microfiche  The  TMDE  Register  contains  approximately  3,000  items,  includ¬ 
ing  calibration  equipment  and  about  110  items  of  ATE  (A  1983  purge  of  the  Register  reduced  the 
number  of  line  items  by  about  40  percent  )  The  3,000  items  include  both  high-densitv  items  used  by 
field  maintenance  units  and  low-density  items  used  at  research  and  development  activities  and 
depots  We  stated  previously  that  the  Army’s  inventory  of  common  ETE  (FSC6625)  includes  over 
18,000  NSNs  (Test  Equipment  Management.  Logistics  Management  Institute,  Task  ML403,  January 
1985)  The  discrepancy  is  explained  by  noting  that  the  TMDE  Register  is  not  an  inventory  listing 
(rather  it  is  defined  as  a  list  of  all  common  TMDE  that  have  received  acquisition  approval  in  the  past), 
that  some  items  carry  multiple  NSNs,  and  that  disposal  of  obsolete  TMDE  has  not  been  policed  As  a 
result,  a  precise  count  of  the  TMDE  inventory  (line  items  and  density)  is  not  available  The  Army  is 
now  in  the  process  of  developing  an  automated  TMDE  inventory  system  and  data  base  (common 
TMDE  only),  with  the  initial  version  scheduled  for  completion  in  late  1985  It  is  also  implementing 
new  regulations  for  turn-in  of  obsolete  TMDE 


Test  equipment  acquisition  in  the  Navy  is  performed  by  numerous  activities  In  this  sec¬ 
tion,  we  focus  on  the  acquisition  of  test  equipment  in  support  of  submarine/surface  ship  acquisition 
programs  managed  by  the  Naval  Sea  Systems  Command  (NAVSEA)  and  that  in  support  of  aircraft 
acquisition  programs  managed  by  the  Naval  Air  Systems  Command  (NAVAIR).  (Test  equipment 
acquisition  procedures  for  the  Navy’s  strategic  systems,  for  industrial  activities  and  laboratories,  and 
for  the  Marine  Corps  are  different  in  various  degrees.) 

NAVSEA.  Test  equipment  recommendations  resulting  trom  the  contractor’s  LSA  efforts 
are  forwarded  by  the  prime  contractor  (normally  the  lead  shipyard)  via  the  cognizant  Supervisor  of 
Shipbuilding,  Conversion,  and  Repair  to  the  ship  acquisition  program  manager  (PMS).  As  in  the  case 
of  the  Army,  contractors  are  normally  required  to  screen  lists  of  existing  test  equipment  in  preparing 
their  test  equipment  recommendations,  with  selection  typically  in  the  following  order  of  priority  for 
NAVSEA  programs: 

•  MIL-STD-1364  (Navy),  "Standard  General  Purpose  Electronic  Test  Equipment” 
(current  version  F,  March  1982)  This  document  lists  the  Navy's  preferred  items  of 
common  ETE.  It  contains  about  350  items.  It  is  maintained  and  updated  by 
NAVELEX  every  3  years,  with  quarterly  updates  ("GPETE  Status  List  With  Refer¬ 
ence  Prices”)  published  and  distributed  within  the  Navy. 

•  NAVSEA-0967-LP-098-9000,  "Test  Equipment  Index  to  Shipboard  Portable  Elec¬ 
tronic  Test  Equipment  Requirements  List  (5PETERL)  ”  This  index  lists  all  ETE 
(common  and  peculiar)  in  the  NAVSEA  inventory,  organized  by  type  of  test  equip¬ 
ment.  Currently,  it  contains  about  550  different  types,  including  140  nonstandard 
GPETE  and  the  rest  special-purpose  test  equipment  (SPETE)  It  is  maintained  and 
updated  by  NAVSEA. 

•  Other  Government/DoD  inventory  lists. 

•  Off-the-shelf  commercial  test  equipment 

•  Modifications  to  existing  test  equipment 

•  Development  of  new  peculiar  test  equipment. 

After  PMS  review  and  approval,  the  test  equipment  recommendations  are  forwarded  to 
the  cognizant  organization  within  NAVSEA  (Weapons  and  Combat  Systems  Directorate,  Test  and 
Monitoring  Systems  Division,  SEA-06C1)  to  validate  the  requirement  and  determine  what  activity 


has  management  responsibility  for  each  item.  (SEA-06C1  is  also  responsible  for  reviewing  and 
approving  the  shipboard  test  equipment  allowance  list.)  The  PMS  is  responsible  for  acquisition  and 
management  of  most,  but  not  all,  SPETE  requirements.  The  GPETE  and  selected  SPETE  require¬ 
ments  are  forwarded  with  an  approval  request  form  to  NAVELEX.  The  cognizant  office  within 
NAVELEX  (ELEX-84)  reviews  the  proposed  test  equipment,  purchases  the  SPETE  that  is  under  its 
management,  determines  the  proper  classification  (general  purpose  versus  special  purpose)  of  the 
remaining  ETE,  and  reevaluates  the  need  for  any  nonstandard  GPETE  (i.e.,  ETE  classified  as  general 
purpose,  but  not  included  in  MIL-STD-1364).  (As  an  aside,  the  Naval  Electronics  System  Engineer¬ 
ing  Activity,  St.  Inigoes,  Maryland,  not  NAVELEX,  provides  the  engineering  support  for  ETE.) 
Based  on  its  review,  NAVELEX  may  decide  to  substitute  standard  items  for  nonstandard  GPETE 
items  specified  in  the  approval  request.  All  GPETE  is  purchased  by  NAVELEX  using  funds  provided 
by  the  PMS.  (If  the  program,  however,  involves  a  ship  or  weapons  system  modification  instead  of  a 
new  system,  NAVELEX  funds  the  GPETE.) 

The  actual  mechanism  for  test  equipment  acquisition  involves  several  field  activities 
and  fund  transfers.  SPETE  purchased  by  the  PMS  is  actually  contracted  for  by  SPCC.  the  latter  may 
also  become  the  inventory  manager  for  those  items  depending  on  the  PMS’s  delegation  of  responsi¬ 
bility.  For  ETE  purchased  by  NAVELEX,  funds  are  transferred  to  the  NAVELEX  Detachment 
Mechanicsburg,  which  then,  in  conjunction  with  NAVELEX  Headquarters,  prepares  the 
documentation  needed  by  SPCC  for  purchasing  the  item.  SPCC  performs  the  contracting  and,  in  most 
cases,  becomes  the  item  manager. 

NAVAIR.  NAVA1R,  like  the  Air  Force,  has  adopted  SISMS  in  managing  the  acquisition 
of  test  equipment.  The  process  is  based  on  contractor  identification  of  test  equipment  requirements 
through  the  weapons  system  LSA  effort  and  documentation  of  contractor-recommended  solutions  in 
the  Support  Equipment  Recommendation  Data  (SERD)  The  SERD  is  a  standard  contractual  data 
item  prepared  for  each  piece  of  support  and  test  equipment  It  includes  a  functional  analysis. 
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recommended  solution,  and,  for  new  test  equipment,  the  necessary  design  and  logistic  support  data. 
The  contractor  is  normally  required  to  use  the  following  order  of  priority  in  selecting  test  equipment. 

•  Preferred  items  listed  in  NAVAIR  PILs: 

-  NAVAIR  16- 1-525,  "Avionics  Preferred  Common  Support  Equipment" 

-  NAVAIR  19-1-127,  "Non-Avionics  Preferred  Common  Support  Equipment” 

•  Existing  peculiar  test  equipment,  using  the  following  data  sources: 

-  Engineering  Data  Retrieval  System,  a  large  data  base  maintained  by  the  Naval 
Air  Engineering  Center  (NAEC),  Lakehurst,  New  Jersey 

-  MIL-HDBK-300  (U  SAF)  "Technical  Information  File  of  Support  Equipment” 

•  Off-the-shelf  commercial  test  equipment 

•  Modification  of  existing  common  or  peculiar  test  equipment 

•  Development  of  new  peculiar  test  equipment. 

NAVAIR  normally  supports  the  contractor  in  the  SERD  development  process  by  providing  a  resident 
support  equipment  expert  from  NAEC  for  liaison  and  data  feedback  from  NAEC’s  data  bases  as 
required  by  the  contractor. 

NAEC  screens  and  verifies  all  completed  SERDs.  It  ensures  that  there  is  a  valid  test 
equipment  requirement;  it  verifies  that  the  SERD  is  complete;  and  it  solicits  comments  on  selected 
SERDs  from  appropriate  activities  (e.g.,  the  cognizant  Naval  Air  Rework  Facility  would  normally 
review  any  depot-level  test  equipment  recommendations).  It  also  reevaluates  any  new  test  equip¬ 
ment  recommendations  by  screening  the  PILs  and  Engineering  Data  Retrieval  System  to  determine 
whether  existing  test  equipment  could  satisfy  the  testing  requirements.  One  of  the  tools  used  by 
NAEC  is  a  formalized  support  equipment  selection  analysis  process  that  is  described  in  Appendix  E. 
If  it  finds  that  new  test  equipment  is  indeed  necessary,  it  performs  a  technical  evaluation  of  the 
contractor’s  recommended  items  Once  its  review  is  completed,  NAEC  prepares  a  SERD  disposition 
letter,  approving,  modifying,  or  disapproving  the  contractor's  SERD 

NAEC  operates  under  the  direction  of  NAVAIR's  centralized  support  equipment  man¬ 
ager,  the  Systems  and  Engineering  Group,  Support  Equipment  Division  (AIR-552),  which  is 
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responsible  for  managing  all  aspects  of  naval  aviation  support  equipment.  AIR-552  normally 
delegates  SERD  approval  for  all  nonmajor  items  to  NAEC.  Its  primary  emphasis  is  on  budget  items; 
it  budgets  support  equipment  requirements  for  the  aircraft  program  manager  and  it  monitors  the 
funds  allocated  for  peculiar  support  equipment  as  well  as  funds  allocated  for  NAVAIR’s  common 
support  equipment.  NAEC  coordinates  funding  requirements  with  AIR-552.  (In  contrast  to  the  other 
Military  Services,  AIR-552  is  the  ultimate  SERD  approval  authority,  not  the  weapons  system 
program  manager.) 

The  approved  SERD  is  forwarded  to  the  Aviation  Supply  Office  and  the  Naval  Air  Tech¬ 
nical  Services  Facility.  The  Aviation  Supply  Office,  the  inventory  control  point  for  aviation  items  and 
associated  peculiar  support  equipment,  computes  allowance  requirements  for  support  equipment, 
provisions  for  spare  and  repair  parts,  and  coordinates  with  other  inventory  managers  to  ensure 
availability  of  items  that  are  not  under  its  control.  Standard  GPETE  is  not  under  the  control  of  the 
Aviation  Supply  Office;  it  is  managed  by  NAVELEX,  with  SPCC  performing  the  day-to-day 
management  control.  The  Naval  Air  Technical  Services  Facility  is  responsible  for  monitoring  the 
preparation  and  delivery  of  technical  manuals,  maintenance  requirements  cards,  calibration  meas¬ 
urement  requirements  summaries,  and  calibration  procedures.  Calibration  measurement  require¬ 
ments  summaries  and  calibration  procedures  are  also  reviewed  and  approved  by  the  Naval  Aviation 
Logistics  Center,  which  is  responsible  for  ensuring  that  adequate  calibration  equipment  will  be  avail¬ 
able  to  satisfy  the  test  equipment  calibration  requirements. 

Peculiar  ETE  is  acquired  generally  by  NAVAIR  directly  via  the  aircraft  contract,  and  its 
development  is  monitored  by  NAEC  Common  ETE  is  acquired  by  forwarding  funded  requisitions  to 
SPCC,  which  then  purchases  the  equipment. 

The  above  outline  illustrates  that  the  "centralized  manager”  for  standard  GPETE 
(ELEX-84)  has  only  limited  control  over  NAVAIR’s  selection  of  test  equipment  MIL-STD-1364  is  not 
referenced  in  SISMS  nor  in  NAVAIR’s  description  of  its  test  equipment  acquisition  management 
process.  SISMS  emphasizes  MIL-HDBK-300  as  the  single  data  source  for  standardization  of  support 
equipment  Some  MIL-STD-1364  items  may  be  contained  in  MIL-HDBK-300,  many  are  not 


2  10 


Air  Force 


The  Air  Force’s  organizational  structure  for  test  equipment  acquisition  management  is 
somewhat  complex.  AFSC  is  responsible  for  the  acquisition  of  new  aircraft  systems  and  all  new 
peculiar  support  equipment,  while  AFLC  is  responsible  for  supplying  the  prime  contractor  with  all 
required  common  support  equipment  as  Government-furnished  equipment  and  for  managing  all 
follow-on  logistic  support.  However,  the  support  equipment  selection  and  approval  process  involves  a 
number  of  organizations  with  different  responsibilities.  While  the  various  product  divisions  of  AFSC 
follow  different  procedures,  the  description  that  follows  is  limited  to  the  largest  product  division  in 
AFSC,  the  Aeronautical  Systems  Division. 

Contractual  guidance  on  the  preparation  of  SERDs  for  a  new  weapons  system  normally 
specifies  the  following  order  of  priority  for  test  equipment  selection: 

•  Equipment  defined  by  existing  Government  specifications  or  modifications  thereof 

•  Off-the-shelf  commercial  test  equipment  already  in  the  Government  inventory 

•  Other  off-the-shelf  commercial  or  modified  equipment 

•  Development  of  new  peculiar  test  equipment. 

The  contractor  is  required  to  screen  MIL-HDBK-300  as  the  primary  data  source  for 
existing  support  equipment  (the  first  two  items  above).  Another  data  source  that  is  frequently  used 
for  ETE  in  the  Air  Force  inventory  is  T  33K-1-100,  "Precision  Measurement  Equipment  Interval, 
Labor  Estimate,  Calibration  and  Repair  Technical  Order  Reference  Guide  and  Work  Unit  Code 
Manual  ”  MIL-HDBK-300  is  maintained  by  AFLC's  Cataloging  and  Standardization  Office,  Battle 
Creek,  Michigan,  and  lists  support  equipment  for  all  aircraft  and  missile  systems  in  the  DoD 
inventory  The  equipment  included  on  that  list  has  a  unit  cost  of  at  least  $1,000  or  a  total 
procurement  value  of  at  least  $100,000  regardless  of  unit  cost.  (The  Cataloging  and  Standardization 
Office  has  had  serious  problems  keeping  this  data  base  up  to  date,  furthermore,  the  data  base  is 
difficult  to  use  because  it  is  published  on  microfiche  )  The  T  O  33K-I-100  is  maintained  bv  San 
Antonio  ALC,  the  item  manager  of  all  common  ETE  (FSC  6625)  in  the  Air  Force 
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Since  the  SERDs  are  reviewed  by  numerous  organizations  in  the  Air  Force,  the  allotted 
time  of  60  days  is  often  insufficient  to  complete  the  review-and-approval  cycle.  The  contractor  sub¬ 
mits  SERDs  normally  to  two  organizations,  the  weapons  system  program  manager  or  system  program 
office  (SPO),  and  the  end-item  system  manager  (located  at  an  ALC). 

Each  weapons  system  SPO  is  supported  by  a  team  of  logistics  experts,  on  a  matrix  organ¬ 
ization  basis,  from  the  Air  Force  Acquisition  Logistics  Center,  an  organization  that  reports  both  to 
AFSC  and  AFLC.  The  team,  headed  by  the  Deputy  Program  Manager  for  Logistics,  plans,  reviews, 
and  manages  all  logistics  areas  of  a  weapons  system  acquisition  program,  including  support  equip¬ 
ment  acquisition  logistics.  In  addition,  the  support  equipment  system  program  office  (SESPO) 
assigns  support  equipment  managers  to  the  weapons  system  SPO  for  the  integration  of  financial 
management  of  support  equipment  acquisition.  The  SESPO  is  a  relatively  recent  organization 
designed  to  improve  the  management  of  support  equipment  in  the  acquisition  process,  integrate  all 
research  and  development  of  support  equipment,  and  increase  commonality  (cross-system  applica¬ 
tion)  of  support  equipment.  Both  the  support  equipment  manager  and  SESPO  review  the  SERD  and 
report  their  findings  or  recommendations  to  the  weapons  system  SPO  through  the  Deputy  Program 
Manager  for  Logistics. 

Within  AFLC,  the  end-item  system  manager  distributes  the  SERD  to  the  following 
organizations  for  review: 

•  The  user  of  the  weapons  system,  such  as  Tactical  Air  Command 

•  The  ALCs  that  will  be  the  item  managers  of  the  support  equipment.  Their  respon¬ 
sibility  is  to  conduct  an  engineering  and  technical  review  to  determine  whether 
existing  support  equipment  or  modifications  thereof  (or,  alternatively,  commercial 
off-the-shelf  test  equipment)  could  satisfy  the  requirement  rather  than  contractor- 
proposed  new  peculiar  equipment. 

•  The  ALCs  that  will  provide  depot-level  maintenance  of  the  support  equipment  (The 
assignment  of  depot-level  maintenance  responsibility  for  each  subsystem  may  not 
necessarily  coincide  with  that  of  the  item  manager.) 

•  The  Aerospace  Guidance  and  Metrology  Center  (AGMC),  which  is  responsible  for 
reviewing  the  Calibration  Measurements  Requirements  Summary  of  new  test 
equipment,  evaluating  the  need  for  new  calibration  equipment,  and  developing 
calibration  procedures  once  the  SERD  is  approved 


•  Cataloging  and  Standardization  Office. 


After  these  organizations  review  the  SERD,  their  recommendations  are  accumulated  by 
the  end-item  system  manager  and  reported  to  the  Aeronautical  Systems  Division  weapons  system 
SPO,  who  has  the  final  approval  authority  on  the  selection  of  support  equipment.  Once  the  decision  is 
made,  items  are  either  acquired  through  the  weapons  system  contract  as  contractor-furnished  equip¬ 
ment  or  through  the  cognizant  ALC  as  Government-furnished  equipment.  Funding  of  the  latter  is 
provided  by  AFLC. 

It  may  be  of  interest  to  draw  a  comparison  between  the  acquisition  management  proc¬ 
esses  of  the  Air  Force  and  NAVAIR.  The  NAVAIR  Support  Equipment  Division  (AIR-552)  is  com¬ 
parable  to  the  Air  Force  Aeronautical  Systems  Division  SESPO  in  that  it  provides  centralized 
management  of  all  support  equipment.  It  is,  however,  different  in  two  respects:  AIR-552,  not  the 
aircraft  program  manager,  has  the  final  approval  authority  for  SERDs;  furthermore,  it  is  ultimately 
responsible  for  life  cycle  management  of  fielded  support  equipment  -  a  responsibility  that  is  dispersed 
among  various  ALCs  in  the  Air  Force.  SESPO,  as  an  AFSC  organization,  is  not  similarly  involved 
Observations 

In  practice,  a  number  of  weak  points  in  the  test  equipment  selection-and-approval  proc¬ 
ess  are  commonly  well  recognized  They  include: 

•  The  process  is  invariably  started  late  in  the  weapons  system  development  cycle, 
which  impedes  tradeoffs  between  design  characteristics  and  support  equipment 
commonality  As  a  result,  opportunities  for  more  standardization  of  test  equipment 
are  lost. 

•  The  screening  of  inventory  lists  by  the  contractor  in  preparing  test  equipment 
recommendations  is  impeded  because  the  lists  are  incomplete,  out-of-date,  and  diffi 
cult  to  access  Furthermore,  the  process  is  not  policed 

•  The  review  an^  .  erification  of  the  contractor's  recommendations  by  the  DoD  is  often 
limited  by  scarce  resources  (experienced  personnel  and  schedule) 

A  perhaps  less  obvious  weakness  is  that  the  entire  process,  by  its  very  nature,  tends  to  be 
suboptimal  because  it  deals  with  test  requirements  in  a  piecemeal  fashion  This  affects  both  the  test 
equipment  required  to  support  a  single  weapons  system  and  the  test  equipment  required  by  a 
maintenance  unit  supporting  multiple  weapons  systems 
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With  regard  to  the  first,  a  single  weapons  system  may  involve  hundreds,  if  not  thou¬ 
sands,  of  SERDs  These  documents  are  prepared  and  reviewed  in  sequence,  not  all  at  the  same  time 
While  all  the  SERDs  may  be  correct,  the  composite  may  not  be  optimal  The  key  factors  in  this  proc¬ 
ess  appear  to  be  the  contractor’s  incentive,  which  often  is  counter  to  DoD’s,  and  the  small  size  of  the 
DoD’s  review  team. 

With  regard  to  the  second,  both  the  Navy  and  Air  Force  have  an  effective  process  for 
developing  a  test  equipment  allowance  list  (minimizing  the  variety  and  density  of  test  equipment)  for 
a  ship  or  aircraft  squadron,  but  the  Army  does  not  have  a  similar  process  for  its  maintenance  units 
(especially  at  the  intermediate  level).  Test  equipment  requirements  for  each  new  system  are  added 
incrementally  to  whatever  test  equipment  the  supporting  maintenance  units  already  are  authorized 
in  their  Tables  of  Organization  and  Equipment.  The  Army’s  Program  Manager  for  TMDE  has 
recognized  this  deficiency  and  recently  instituted  a  new  approach  to  test  equipment  requirements 
determination 

It  is  noteworthy  that  the  Military  Services  use  different  lists  in  screening  the  DoD 
inventory  of  test  equipment  and  apply  different  criteria  in  selecting  candidate  items  To  illustrate, 
the  Army  and  Air  Force  prefer  modifying  existing  military  specification  equipment  over  procuring 
commercial  off-the-shelf  test  equipment;  in  contrast,  the  Navy  (both  NAVSEA  and  NAVAIR)  gives 
priority  to  commercial  off-the-shelf  equipment  over  modification  of  existing  common  or  peculiar  test 
equipment 

An  additional  comment:  MIL-STD-680A,  "Contractor  Standardization  Program 
Requirements,”  2  October  1981,  is  approved  for  use  by  all  DoD  agencies  and,  if  invoked  in  the 
contract,  requires  contractors  to  exercise  standardization  discipline  throughout  each  weapons  system 
acquisition  phase.  With  respect  to  tools  and  support  equipment,  it  requires  the  same  controls  as 
applied  to  system  components  and  equipment 

Commonality  shall  be  stressed  and  multi-application  tools  and  support 
equipment  shall  be  used  wherever  possible.  The  contractor  shall  review  the 
tools  and  support  equipment  currently  in  the  Government  inventory  for 
application  to  the  contract.  On  the  basis  of  such  review,  the  contractor 


shall  use  current  Government  tools  and  support  equipment  in  preference  to 
the  introduction  of  new  tools  and  support  equipment  wherever  possible. 

However,  a  Navy-peculiar  appendix  to  the  standard  enumerates  the  "design  selection  lists”  for  use  in 
the  contractor’s  standardization  program,  including  N’AVSHIPS-0969-019-7000,  "Electrical  Test 
Equipment  Application  Guide”  for  ETE;  MIL-HDBK-300,  "Technical  Information  File  of  Support 
Equipment”  for  aerospace  ground  support  equipment;  MIL-STD-1364,  "Standard  General  Purpose 
Electronic  Test  Equipment”;  NAVAIR  16-1-525,  "Avionics  Preferred  Common  Support  Equipment”; 
and  NAVAIR  19-1-527,  "Non-Avionics  Preferred  Common  Support  Equipment.” 

A  final  comment  pertains  to  the  feasibility  of  changing  the  acquisition  process  by  con¬ 
straining  weapons  system  design  to  be  supportable  with  a  given  list  of  standard  test  equipment.  Such 
a  requirement  has  never  been  imposed  except  for  standard  ATE.  Several  acquisition  programs  have 
specified  that  the  system  be  supportable  with  a  given  ATE,  but  that  approach  has  not  been  very 
successful.  Many  acquisition  programs,  especially  those  in  the  Army,  include  a  requirement  to  mini¬ 
mize  the  need  for  test  equipment  at  the  organizational  level.  Such  requirements  can  be  successful  in 
directing  the  design  toward  comprehensive  built-in  test.  However,  even  when  contract  requirements 
are  clearly  specified,  success  is  not  guaranteed.  The  best  example,  probably,  is  the  UH-60  Black 
Hawk  program,  where  the  specific  tools  used  by  organizational  maintainers  were  nailed  to  the  wall  in 
the  contractor’s  design  shop  as  a  visible  reminder  for  the  design  engineers  to  constrain  their  design  to 
be  supportable  with  those  tools.  This  innovative  and  highly  effective  approach  was  large!;,  successful, 
even  though  it  became  obvious  once  the  UH-60  was  fielded,  that  a  special  test  set  was  necessary  to 
cope  with  the  electronic  control  unit  assembly  of  the  turbine  engine 

The  UH-60  example,  however,  does  illustrate  that  support  equipment  commonality  is 
achievable  if  it  receives  sufficient  management  attention  at  an  early  stage  when  it  counts  Proper 
implementation  of  "front-end  analysis,”  as  prescribed  in  the  1983  revision  of  MIL-STD-1388,  may- 
help  to  achieve  better  tradeoffs  in  the  future  between  weapons  system  design  characteristics  and  sup¬ 
port  equipment  commonality 


PROCUREMENT  OF  COMMON  ETE 


The  result  of  the  acquisition  management  process  described  above  is  the  development  of  pecul¬ 
iar  test  equipment  and/or  the  procurement  of  common  test  equipment.  With  regard  to  new  common 
ETE,  the  trend  is  toward  the  procurement  of  commercial  off-the-shelf  test  equipment  for  which  the 
Military  Services  (with  the  exception  of  the  Marine  Corps)  have  adopted  a  similar  procurement 
approach,  known  as  "bid  sample  testing.” 

Bid  sample  testing  is  a  two-step  competitive  procurement  method  in  which  the  first  step  entails 
a  technical  evaluation  of  the  candidate  products  and  the  second  step  consists  of  soliciting  bids  from 
those  vendors  who  passed  the  evaluation.  This  method  permits  the  DoD  to  meet  its  need  for  a  product 
at  the  lowest  cost  when  the  characteristics  of  that  product  cannot  be  adequately  described  in  a 
detailed  specification  or  purchase  description. 

The  Federal  Acquisition  Regulation  (FAR)  is  clear  about  the  Government’s  policy  favoring 
commercial  products  whenever  possible: 

In  a  manner  consistent  with  statutes,  Executive  Order,  and  governing  reg¬ 
ulations  regarding  maximum  competition,  agencies  shall  acquire  commer¬ 
cial  products  and  use  commercial  distribution  systems  whenever  these 
products  or  distribution  systems  adequately  satisfy  the  Government’s 
needs.  [FAR,  section  11.002] 

The  FAR  is  also  clear  about  the  circumstances  under  which  bid  sample  testing  is  appropriate: 

(1)  Bidders  shall  not  be  required  to  furnish  bid  samples  unless  there  are 
characteristics  of  the  product  that  cannot  be  described  adequately  in  the 
specification  or  purchase  description. 

(2)  Bid  samples  will  be  used  only  to  determine  the  responsiveness  of  the 
bid  and  will  not  be  used  to  determine  a  bidder’s  ability  to  produce  the 
required  items. 

(3)  Bid  samples  may  be  examined  for  any  required  characteristic, 
whether  or  not  such  characteristic  is  adequately  described  in  the  specifica¬ 
tion,  if  listed  in  [the  invitation  for  bid  |. 

(4)  Bids  will  be  rejected  as  nonresponsive  if  the  sample  fails  to  conform 
to  each  of  the  characteristics  listed  in  the  invitation.  [FAR,  para 
graph  14  202-4(b)l 
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All  Military  Services  use  MIL-T-28800C,  "Test  Equipment  For  Use  with  Electrical  and 
Electronic  Equipment,  General  Specification  for”;  23  December  1981,  in  preparing  their  purchase 
description.  That  document  also  specifies  the  tests  and  inspections  for  quality  assurance  of  the 
candidate  products.  A  brief  description  of  the  general  specification  and  the  procedures  of  the  Military 
Services  follows. 

General  Specification 

Although  MIL-T-28800C  describes  the  general  requirements  for  test  equipment,  it  is  not 
suitable  to  be  invoked  in  a  contract  without  the  use  of  a  detailed  specification,  such  as  a  military 


specification,  purchase  description,  or  a  "brand  name  or  equal”  purchase  description. 

It  categorizes  test  equipment  by  type  (design/construction),  class  (environmental 
requirements),  style  (type  of  enclosure),  and  color.  There  are  three  types  (Type  I  is  designated  for 
military  use,  Type  II  is  modified  commercial  off-the-shelf  equipment,  and  Type  III  is  commercial  off- 
the-shelf);  five  classes  (with  different  environmental  conditions  such  as  temperature,  humidity,  and 
shock  resistance);  nine  styles  (ranging  from  extremely  rugged  cases  to  rack-mount  cases  and  console 
cabinets);  and  eleven  colors.  MIL-T-28800C  describes  the  requirements  for  safety,  parts  and 
materiels,  design  and  construction,  electrical  power  sources  and  connections,  mechanical  stability, 
enclosures,  marking,  environmental  conditions,  reliability,  maintainability,  and  interchangeability 
As  an  example,  for  Type  III  reliability,  it  states  that  unless  a  mean  time  between  failures  (MTBF)  is 
specified  in  the  detailed  specification,  the  upper  test  MTBF  for  continuous  and  intermittent  operation 
shall  be  1,500  hours  under  the  environmental  conditions  specified  for  the  equipment,  when  tested  in 
accordance  with  MIL-STD-781,  test  plan  IlIC.  Similarly,  for  Type  III  maintainability,  it  states  that, 
unless  otherwise  required  in  the  detailed  specification,  equipment  design  shall  permit  isolation  of 
faults  down  to  the  lowest  discrete  component  with  the  maintenance  provisions  furnished  as  part  of 
the  equipment.  A  maintainability  demonstration,  if  required  in  the  contract,  shall  be  performed  in 


accordance  with  MIL-STD-471,  with  a  mean  corrective  maintenance  time  of  90  minutes,  maximum 
tolerable  mean  corrective  maintenance  time  of  115  minutes,  and  maximum  corrective  maintenance 


time  not  to  exceed  270  minutes.  With  respect  to  calibration  requirements,  it  states  that  the 
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calibration  interval,  unless  otherwise  specified  in  the  detailed  specification,  shall  be  9  months,  based 
on  an  equipment  operation  time  of  1,500  hours. 

MIL-T-2880CC  also  lists  the  tests  or  quality  assurance  provisions  for  the  three  types  of 
inspection  conducted  by  suppliers  and/or  the  Government:  first  article  inspection,  bid  sample  inspec¬ 
tion,  or  quality  conformance  inspection.  The  tests  or  inspections  are  categorized  into  six  groups. 

•  Group  A.  Preoperational  inspection  (materials,  workmanship,  and  safety  character¬ 
istics,  including  random  vibration  and  temperature  cycling  tests);  Level  A  perform¬ 
ance  test  (exercising  all  functions  and  modes  of  operation);  open  circuit  voltage  test; 
and  leakage  current  test 

•  Group  B.  Level  B  performance  test  in  accordance  with  approved  equipment  test  pro¬ 
cedures  or  the  detailed  specifications 

•  Group  C.  Environment  tests,  including  temperature  and  humidity,  altitude,  vibra¬ 
tion,  shock,  water  resistance,  and  electrical  power  tests  (power  consumption,  voltage 
and  frequency  variation,  and  transients) 

•  Group  D.  Environmental  tests,  including  fungus  resistance,  salt  atmosphere,  explo¬ 
sive  atmosphere,  sand  and  dust,  electromagnetic  interference,  and  magnetic  envi¬ 
ronment  tests 

•  Group  E.  Miscellaneous  tests  and  inspections,  including  maintainability,  acoustic 
noise,  dimensions  and  weight,  mechanical  stability,  equipment  emanations,  front 
panel  markings,  and  packaging 

•  Group  F.  Reliability  tests. 

For  bid  sample  testing,  the  specification  states  that,  unless  otherwise  required  in  the 
solicitation,  the  bid  sample  shall  consist  of  two  equipments  with  accessories  and  instruction  manuals 
and  "clear  and  concise”  rationale  showing  how  the  reliability  and  maintainability  (R&M)  characteris¬ 
tics  of  the  equipment  comply  with  the  requirements  of  the  solicitation  (hereafter  referred  to  as  the 
"R&M  rationale”).  It  further  specifies  that  the  two  bid  sample  equipments  will  be  subjected  first  to 
the  Group  A  inspections  and  then  to  the  Group  B  tests  (if  required)  if  both  pass  the  Group  A  inspec¬ 
tions  Next,  one  bid  sample  equipment  will  be  subjected  to  Group  C  and  Group  D  tests,  and,  finally, 
both  equipments  will  be  subjected  to  Group  E  tests.  Importantly,  the  specification  stipulates  that, 
unless  otherwise  specified  in  the  solicitation,  maintainability  and  reliability  testing  will  not  be  per¬ 
formed  on  bid  samples. 
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Army.  The  Project  Manager  for  TMDE  Modernization  (PM,  TEMOD),  as  the  acquisition 
manager  for  new  common  ETE,  and  CECOM,  as  the  item  manager  of  the  Army’s  common  ETE,  have 
used  the  competitive  bid  sample  approach  to  procure  commercial  off-the-shelf  test  equipment  since 
1981.  The  Army  issues  a  letter  request  for  bid  samples  with  a  purchase  description  specifying  the 
functional  performance  requirements  and  identifying  the  "facility  of  use”  characteristics  (adopted 
from  MIL-T-28800C)  that  will  be  evaluated  in  the  test.  The  Army  usually  requires  a  sample  of 
three  instruments:  one  for  performance  testing  (with  a  maximum  of  three  failures  allowed)  and 
two  for  "facility  of  use”  testing.  The  facility  of  use  testing,  conducted  by  several  groups,  focuses  on 
eight  characteristics:  safety,  workmanship,  human  factors,  construction/design,  application  compat¬ 
ibility,  tester  portability,  ease  of  calibration,  and  maintainability.  Evaluation  and  test  reports  are 
reviewed  by  CECOM  and  PM,  TEMOD  personnel,  with  final  approval  authority  by  PM,  TEMOD. 

Vendors  that  pass  the  bid  sample  test  receive  an  invitation  for  bid.  Bids  are  evaluated 
on  the  basis  of  three  cost  elements:  (1)  hardware  (unit  cost  and  initial  provisioning,  which  are  priced 
separately  by  the  bidder);  (2)  documentation  (separately  priced  in  the  bid),  and  (3)  initial  training  (if 
required  and  separately  priced).  The  lowest-cost  bidder  receives  the  contract. 

In  1983,  the  Army  examined  the  life  cycle  costs  of  four  items  of  common  ETE  to  identify 
cost  categories  that  have  a  significant  contribution  to  life  cycle  cost,  differentiate  between  bidders, 
and  are  verifiable  in  bid  sample  testing  It  found  that  four  cost  elements  met  all  three  conditions:  test 
equipment  hardware  cost  (57  to  69  percent  of  life  cycle  cost),  maintenance  and  calibration  labor  cost 
(4  to  17  percent),  consumables  (8  to  11  percent),  and  initial  provisioning  of  spare  parts  (5  to  7  percent). 
Based  on  these  findings,  the  Army  decided  to  include  an  estimate  of  maintenance  and  calibration 
labor  cost  in  its  bid  evaluation  and  to  improve  its  estimate  of  initial  provisioning  costs  (rather  than 
relying  on  vendor  estimates). 
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In  fiscal  year  1985  (FY85),  the  Army  began  using  a  simplified  cost  model  to  estimate 
ownership  costs  from  the  following  input  data  (with  the  first  four  parameters  specified  by  the  Army 
for  a  particular  procurement). 

•  Life  cycle  (years  of  operation) 

•  Quantity  (number  of  items  required) 

•  Utilization  (number  of  operating  hours  per  year) 

•  Hourly  labor  wage  (depot-level  technicians) 

•  Calibration  interval  (vendor  specified) 

•  MTBF  (vendor  specified) 

•  Mean  time  to  repair  (MTTR). 

The  solicitation  package  further  includes  a  minimum  acceptable  MTBF  and  a  maximum 
acceptable  MTTR.  In  the  bid  sample  testing,  each  bidder’s  claimed  MTTR  is  verified  through  a  main¬ 
tenance  demonstration.  The  claimed  MTBF  is  verified  only  for  the  winning  bidder  through  a 
reliability  test.  This  life  cycle  costing  approach  was  for  the  first  time  applied  in  1985  in  the 
procurement  of  a  (commercial)  distortion  analyzer.  Among  the  three  bidders  submitting  bid  samples, 
some  of  the  above  factors  showed  a  wide  range.  The  contract  was  awarded  to  the  bidder  with  lowest 
projected  life  cycle  cost.  Of  the  losing  bidders,  one  had  the  lowest  unit  cost,  the  other  had  the  highest 
reliability.  The  Army  is  planning  to  further  refine  this  procedure,  making  the  demonstrations  a 
separately  priced  option  that  may  not  be  exercised  when  those  factors  (calibration,  reliability,  and 
maintainability)  are  close  and  do  not  influence  contract  award. 

Navy.  NAVELEX,  as  the  centralized  manager  of  Navy  common  standard  ETE,  has  used 
the  competitive  bid  sample  approach  since  1977  Bid  samples  are  required  for  all  competitive  pro¬ 
curements  of  common  test  equipment  by  NAVELEX  (about  75  percent  of  all  its  test  equipment 
funds).  NAVELEX  prepares  a  tailored  purchase  description  using  MIL-T-28800C,  the  specific  meas¬ 
urement  requirements,  and  engineering  judgment,  with  the  "salient  characteristics”  normally 
expressed  as  a  "brand  name  or  equal”  specification  ft  requests  two  samples  per  item  bid  for  testing. 
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Tests  are  conducted  and  evaluated  at  one  of  the  two  Navy  laboratories  in  accordance  with  the  detailed 
purchase  description  and  MIL-T-28800C.  Final  decisions  on  the  test  evaluations  are  made  by 
NAVELEX.  The  second  step  of  the  procurement  (invitation  for  bid  and  contract  award)  is  performed 
by  SPCC.  The  contract  is  awarded  on  the  basis  of  lowest  unit  cost.  NAVELEX  is  considering  some 
type  of  life  cycle  costing  approach  and  is  also  planning  to  put  increased  emphasis  on  the  quality  of 
technical  manuals  in  the  bid  evaluation  process. 

NAVELEX  has  experimented  with  life  cycle  costing  approaches,  but  it  has  not  been 
satisfied  with  the  results.  It  does  not  believe  that  the  Army’s  approach  to  detailed  life  cycle  cost 
analysis  in  procuring  common  ETE  is  a  cost-effective  endeavor,  except  for  very  large  procurement 
programs.  It  finds  the  Army’s  procedure  complex  and  partly  subjective;  particularly,  it  notes  that  the 
projected  service  life  (a  very  subjective  and  arbitrary  estimate)  can  have  a  serious  skewing  effect  on 
any  life  cycle  costing  approach.  Instead,  NAVELEX  has  recently  developed  an  innovative  approach 
that,  in  essence,  levies  the  life  cycle  costing  problem  on  the  vendor.  The  approach  rolls  the  support 
costs  into  the  procurement  contract  through  the  concept  of  a  life  cycle  warranty  or  service  contract. 
For  details  on  this  approach,  which  has  not  yet  been  approved  as  official  Navy  policy,  refer  to 
Appendix  F.  NAVELEX  plans  to  conduct  a  pilot  application  of  this  approach  in  1986  Reactions  to 
the  concept  have  been  mixed:  those  from  large  test  equipment  manufacturers  have  been  most  favor¬ 
able,  those  from  small  manufacturers  and  small  business  advocates  have  been  negative. 

NAVAIR  has  recently  experimented  with  life  cycle  cost  models  in  the  replacement  of 
commercial  test  equipment  embedded  in  ATE  For  example,  the  AN/AWM-23-  which  is  a  hybrid, 
special-purpose  ATE  comprising  five  stations  for  testing  of  line  replaceable  units  from  the 
AN/AWG-9  radar  on  the  F-14A  aircraft  —  is  being  modernized,  primarily  because  the  commercial  test 
equipment  has  become  obsolete  and  unsupportable.  NAVAIR  is  using  a  competitive  procurement 
based  on  technical,  supportability,  and  life  cycle  cost  factors  to  replace  these  items. 

Marine  Corps  The  Marine  Corps  has  never  used  oid  sample  testing  as  a  procurement 
procedure  for  commercial  ETE  Instead,  it  issues  a  detailed  specification  (performance  requirements 
and  "salient  characteristics”  from  MIL-T-28800C)  with  a  request  for  quotation  and  awards  a  contract 
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on  the  basis  of  lowest  price.  (Most  of  its  peculiar  test  equipment  is  acquired  in  the  course  of  weapons 
system  acquisition  programs  managed  by  another  Military  Service,  although  the  number  of  Marine 
Corps  programs  has  grown  in  recent  years.)  In  special  cases,  such  as  for  automated  testers,  it  has 
conducted  some  first  article  test  and  evaluation.  To  meet  emergency  requirements,  it  may  purchase 
from  the  General  Services  Administration  schedule.  Apart  from  these  exceptions,  the  Marine  Corps 
purchases  through  open  bids  without  bid  sample  testing,  relying  on  vendor  claims  of  meeting  the 
detailed  specifications  and  awarding  the  contract  to  the  lowest  bidder. 

Air  Force.  San  Antonio  ALC  adopted  the  competitive  bid  sample  approach  in  the  late 
1970’s  as  its  standard  procedure  for  the  procurement  of  commercial  off-the-shelf  ETE  (FSC  6625). 
Project  engineers  prepare  purchase  descriptions  for  needed  test  equipment  and,  when  funds  are  avail¬ 
able,  item  managers  write  purchase  requests  that  result  in  the  contracting  officer  issuing  solicita¬ 
tions  for  bid  samples.  Vendors  have  45  calendar  days  to  submit  a  bid  sample  package  that  normally 
consists  of  two  instruments,  the  manufacturer’s  operation  and  maintenance  manual,  acceptance  test 
procedures,  and  the  R&M  rationale.  An  Air  Force  test  laboratory  receives  the  bid  sample  package, 
performs  the  inspections  and  tests  in  accordance  with  MIL-T-28800C,  and  writes  the  test  report 
Each  bid  sample  package  is  allowed  two  failures.  After  the  first  failure,  the  tests  assigned  to  the 
failed  instrument  are  reassigned  to  the  other  instrument.  After  the  second  failure,  testing  stops  and 
the  vendor  is  called  in  to  repair  one  of  the  two  failed  instruments.  Testing  then  resumes  on  the 
repaired  instrument.  A  third  failure  during  testing  disqualifies  the  bid  sample 

The  project  engineer  evaluates  the  laboratory’s  test  report  and  either  approves  or  dis¬ 
approves  the  bid  sample  package.  The  entire  process  is  completed  within  60  days  from  receipt  of  the 
package.  The  contracting  officer  issues  an  invitation  for  bid  to  those  vendors  who  passed  the  test,  and 
the  contract  is  awarded  to  the  lowest  bidder 
Observations 

The  overall  approach  to  the  competitive  procurement  of  common  ETE  is  similar  among 
the  Military  Services,  but  the  detailed  procedures  are  different  All,  except  the  Marine  Corps,  are 
trying  to  improve  their  procedures  to  include  evaluation  of  the  cost  of  ownership  A  clear  potential 


exists  for  cost  savings  by  procuring  on  the  basis  of  minimum  life  cycle  cost  instead  of  procurement 
cost.  The  Military  Services  need  to  proceed  with  collecting  and  analyzing  cost  data  to  make  life  cycle 
costing  feasible.  Similar  data  are  needed  also  to  determine  the  most  cost-effective  maintenance 
concept  for  new  test  equipment. 

Furthermore,  the  opportunity  for  significant  cost  savings  through  Joint  Service  procure¬ 
ment  of  common  ETE  (elimination  of  duplicative  testing,  quantity  discounts,  lower  logistic  support 
costs)  is  impeded  by  the  lack  of  Joint  Service  standardization  of  common  ETE  and  common  evaluation 
procedures.  As  a  result,  Joint  Service  buys  are  rare. 

The  life  cycle  warranty  concept  proposed  by  NAVELEX  is  easily  the  most  exciting  idea 
that  has  surfaced  in  the  test  equipment  community  in  recent  years.  It  would  alleviate  most  of  the 
problems  encountered  in  commercial  test  equipment  procurement  and  support;  and  it  should  result  in 
substantial  life  cycle  cost  savings.  Some  refinements  in  the  proposed  approach  may  be  desirable, 
however,  to  protect  against  emergency  and  wartime  support  requirements. 

DEVELOPMENT  OF  STANDARD  ATE 

Confronted  with  escalating  costs  for  ATE,  each  of  the  Military  Services  has  embarked  on  an 
ATE  standardization  program.  Those  programs  are  described  in  this  section 
Air  Force 

In  the  tactical  Air  Force,  field-level  ATE  is  primarily  found  within  the  avionics  interme¬ 
diate  shop  (AIS).  The  AIS,  normally  organic  to  the  wing  it  supports,  provides  intermediate  mainte¬ 
nance,  i.e.,  checkout  and  repair  of  line  replaceable  units  (LRUs),  which  are  assemblies  (boxes) 
removed  from  the  aircraft  by  organizational  maintenance  (Checkout  and  repair  of  shop  replaceable 
units  (SRC's),  which  are  circuit  cards  and  electronic  modules  such  as  power  supplies,  are  depot 
maintenance  functions  in  the  Air  Force.)  The  Air  Force  plans  to  deploy  Tactical  Air  Wings  to  two 
main  operating  bases  in  wartime  Then,  the  A!'"’  will  be  split  into  two  parts,  one  supporting  a  single 
squadron  at  one  base  and  the  second  supporting  the  other  two  squadrons  at  the  second  base  Even 
within  the  tactical  air  community,  there  are  significant  differences  between  L’  S  Air  Forces  in 
Europe  (CSAFE)  and  Pacific  Air  forces  (PACAF)  with  regard  to  intermediate  maintenance  In 


PACAF,  intermediate  maintenance  is  consolidated  at  the  Pacific  Logistics  Support  Center,  Kadena 
Air  Base,  Okinawa,  Japan.  As  a  result,  AISs  are  no  longer  organic  to  the  wings  and  squadrons  as 
they  are  in  USAFE.  Moreover,  since  1984,  PACAF  has  moved  depot-level  maintenance  of  avionics 
forward  by  establishing  the  Support  Center  Pacific,  collocated  with  the  Pacific  Logistics  Support 
Center.  This  is  currently  the  only  field  location  in  the  Air  Force  where  both  LRUs  and  SRL's  are 
tested  on  ATE. 

The  operational  environment  and  mission  requirements  are  characterized  by  the  A1S. 
Each  AIS  supports  a  particular  type  of  aircraft  and  operates  from  a  large  air  base  with  little  con¬ 
straints  on  space  or  other  support  resources  (e  g.,  power,  air  conditioning),  although  for  deployment 
purposes  the  ATE  must  be  air-transportable. 

To  control  ATE  acquisition  and  support  costs,  the  Air  Force  needed  an  approach  to  ATE 
standardization  that  would  permit:  (1)  tailoring  the  ATE  to  the  specific  items  supported  by  each  AIS; 
(2)  standardizing  the  ATE  building  blocks  both  across  different  AISs  and  across  different  mainte¬ 
nance  levels  (intermediate  and  depot);  (3)  utilizing  nonmilitarized,  commercial  test  equipment;  and 
(4)  enhancing  ATE  capabilities  to  support  rapid  avionics  technology  upgrades  without  major  redesign 
of  ATE  hardware  and  software  The  modular  automatic  test  equipment  (MATE)  program  is  designed 
to  accomplish  each  of  these  objectives  The  MATE  program  is  based  on  a  standard  ATE  architecture, 
standard  interfaces,  and  software  standards,  without  actually  standardizing  the  ATE  hardware  com 
ponents  (as  long  as  they  are  M  ATE-qualified  by  the  Air  Force) 

MATE  was  developed  by  a  separate  program  office  in  AFSC,  with  life  cycle  management 
responsibility  by  San  Antonio  ALC,  which  also  operates  the  MATE  Qualification  Center  The  pro¬ 
gram  completed  full-scale  development  this  year  after  a  thorough  operational  test  and  evaluation  of 
MATE  applications  at  both  intermediate  maintenance  (the  intermediate  automatic  test  set  for  the 
A  10  inertial  navigation  system)  and  depot  mair  tenance  (depot  automatic  test  system  for  avionics  for 
the  testing  of  SRL’s  from  a  variety  of  aircraft) 
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The  operational  environment  and  mission  requirements  for  ATE  in  support  of  naval  air 
and  surface  combat  systems  are  different  from  those  of  the  Air  Force.  (Attack  submarines  have  no  off¬ 
line  ATE  aboard.)  Aboard  the  aircraft  carrier,  the  Aircraft  Intermediate  Maintenance  Department 
provides  intermediate  maintenance  to  all  deployed  air  squadrons  and  detachments.  Its  avionics 
branch  employs  a  wide  variety  of  ATE  to  perform  both  LRU  (in  Navy  terminology,  weapon  replace¬ 
able  assemblies  or  WRAs)  and  SRU  (shop  replaceable  assemblies  or  SRAs)  checkout,  fault  isolation, 
and  repair.  The  workload  aboard  each  carrier,  however,  is  similar.  The  operational  environment  also 
differs  from  that  for  the  Air  Force’s  AIS  in  that  space  is  very  cramped  and  conditions  are  less  benign 
(constant  vibration,  unreliable  power  and  air  conditioning). 

Although  the  Navy,  as  an  interim  measure  to  stem  ATE  proliferation,  already  had 
adopted  a  standard  family  of  ATE,  NAVAlR’s  Consolidated  Support  System  program  was  established 
in  1980  as  a  long-range  solution  with  the  objective  of  maximizing  throughput  capability  within  con¬ 
strained  space  by  usinga  flexible  system  architecture  with  standardized  ATE  hardware  and  software 
modules.  Design  concepts  include  integrating  the  ATE  with  automated  information  tools  (mainte¬ 
nance  history,  job  shop  scheduling)  to  increase  the  effective  use  of  the  ATE. 

The  Consolidated  Support  System  program  completed  the  problem  definition  phase  in 
August  1981  and  systems  definition  phase  in  August  1983  Following  a  program  review  by  the  Secre¬ 
tary  of  the  Navy  in  January  1985,  the  program  scope  was  expanded  to  include  the  intermediate-  and 
depot-level  requirements  of  all  Systems  Commands,  and  the  program  name  was  changed  to  Consoli¬ 
dated  Automated  Support  System  (CASS).  The  program,  previously  managed  by  AIR-552,  is  now 
managed  by  a  separate  NAVAIR  program  office  (APC-206),  and  is  in  full-scale  development  with  con¬ 
tracts  awarded  in  September  1985 

Aboard  surface  combatants,  combat  systems  maintenance  was  performed  without  off¬ 
line  ATE  until  recently.  Maintenance  policy  was  (and  still  is)  shipboard  replacement  of  the  lowest 
replaceable  unit  (circuit  cards  and  other  electronic  modules)  and  off-ship  repair  of  that  unit,  usually 
at  a  depot.  Surface  ship  combat  systems  were  designed  accordingly,  relying  on  built-in  test  to  guide 


the  technician  toward  the  faulty  unit  and  requiring  manual  troubleshooting,  using  common  ETE,  to 
fault  isolate  within  ambiguity  groups.  Because  of  limitations  of  built-in  test  and  limited  experience 
and  training  of  Navy  technicians,  this  policy  resulted  in  excessive  removals  and  replacements  of  good 
modules  (60  to  70  percent  of  the  digital  circuit  cards  received  at  depots  for  repair  showed  no  evidence 
of  failure),  depletion  of  the  supply  system,  and  combat  systems  frequently  down  for  lengthy  periods 
awaiting  parts  and  maintenance.4  To  solve  this  problem,  NAVSEA  determined  that  it  needed  to 
place  small,  portable  ATE  aboard  surface  combatants  to  permit  testing  of  circuit  cards  for  two 
purposes:  ( 1)  to  stop  evacuating  good  circuit  cards  to  depots  by  screening  them  (go/no-go  testing),  and 
(2)  to  fault  isolate  (diagnostic  testing)  and  repair  circuit  cards  when  cost  effective  Under  the  Support 
and  Test  Equipment  Engineering  Program,  digital  circuit  card  testers  have  been  installed  aboard 
major  surface  combatants  (cruisers/destroyers)  as  well  as  at  intermediate  maintenance  facilities 
(tenders  and  shore  intermediate  maintenance  activities).  The  ATE  selected  for  this  purpose  by 
NAVSEA,  in  coordination  with  NAVELEX,  was  the  AN/USM-465,  a  modification  of  the  commercial 
off-the-shelf  GENRAD-2225  This  ATE  is  also  used  in  the  Army  and  Marine  Corps  for  the  same 
purposes.  A  successor  to  the  AN/USM-465,  with  expanded  capabilities,  is  under  consideration  and  it 


may  be  acquired  sometime  in  the  future  under  the  CASS  umbrella  program 
Army 

Chronologically,  the  Army’s  need  for  field-level  ATE  was  more  recent  than  that  of  the 
Air  Force  and  Navy.  (For  that  reason,  we  are  describing  the  Military  Services'  ATE  programs  in  a 
sequence  other  than  seniority.)  In  1971,  CECOM  procured  the  first  prototype  of  the  fiilectronics 
Quality  Assurance  Test  Equipment  (EQUATE).  The  EQUATE  (type-designated  AN/USM  410(V)| 
was  an  integration  of  commercial  off-the-shelf  equipment  into  a  general-purpose  ATE  system  for 
fixed-plant  installation.  (Its  testing  capabilities  were  primarily  based  on  meeting  a  depot  testing 


4For  a  detailed  description  of  maintenance  performance  problems  and  contributing  causes  in 
the  Navy,  see  F  Nauta,  Alleviating  Fleet  Maintenance  Problems  Through  Maintenance  Training 
and  Aiding  Research,  Technical  Report  (Orlando,  Florida  Naval  Training  Equipment  Center.  May 
1985) 


requirement  for  radio  end  items  and  subassemblies.)  After  several  iterations,  the  required  opera¬ 
tional  capability  (ROC)  for  General  Support-Automated  Test  Support  Systems  was  approved  in 
March  1979.  It  identified  the  AN/USM-410(  V)  as  a  core  system  for  general-support-level  ATE,  to  be 
augmented  as  needed  for  the  supported  weapons  systems  and  to  be  deployed  in  vans  to  provide  envi¬ 
ronmental  conditioning  of  the  commercial  equipment.  Subsequently,  AMC  Headquarters  decided  in 
December  1979  to  adopt  the  AN/USM-410(V)  as  the  single  Army  standard  ATE  for  general  support 
and  depot  maintenance.  It  issued  implementation  instructions  in  June  1980,  outlining  the  ground 
rules  for  determining  whether  or  not  system-peculiar  ATE,  under  development  or  planned,  should  be 
terminated  in  favor  of  the  AN/USM-410(V).  Since  that  time,  14  major  weapons  system  programs, 
including  Patriot,  have  received  waivers  from  using  the  A\7L'SM-410(  V). 

The  van-mounted  field  configuration  of  the  AN7USM-410(  V),  the  Electronic  Test 
Facility  (ETF)  (OQ-290),  is  equipped  with  the  AN7USM-465  digital  card  tester  and  common  manual 
TMDE.  It  is  deployed  in  combination  with  the  Equipment  Repair  Facility  (ERF),  OA-8991,  which 
houses  a  countertop  workbench  with  four  repair  stations,  an  oven  for  conformal  coating  after  repair, 
and  a  quality  assurance  station  with  microscope.  Each  of  the  two  vans  is  a  35-foot,  8-wheel,  10-ton 
capacity  semitrailer  that  can  be  towed  across  unimproved  roads  with  a  maximum  speed  of  5  miles  per 
hour  but  is  neither  transportable  by  rail  (the  AN/USM-410(  V)  does  not  have  the  required  shock 
resistance)  nor  by  helicopter  (the  AN/USM-4KH  V)  is  too  heavy).  The  whole  configuration,  ERF  plus 
ETF,  nomenclature  AN/MSM-105(V)1,  with  the  requisite  trailer-mounted  electric  power  generator  is 
air  transportable  but  represents  a  full  load  for  the  C-5A.  In  other  versions  of  the  AN/MSM-105,  such 
as  the  AN/MSM-105(V)2  for  the  AH-64  attack  helicopter,  the  ETF  is  augmented  with  system- 
peculiar  ATE  and  housed  in  more  than  one  van,  although  the  ERF  remains  the  same  5 

5A  more  detailed  chronology  of  the  A.\7USM-410(  V),  its  operational  characteristics,  and  the 
augmented  version  AN/MSM- 105( V)2  is  provided  in  Frans  Nauta,  AH-64  Automatic  Test 
Equipment  Requirements.  Working  Note  ML213-1  (Bethesda,  Maryland:  Logistics  Management 
Institute,  November  1982)  (AD  A122879) 


At  the  end  of  FY84,  the  Army  had  on  hand  or  under  contract  142  AN/USM-410(  V)’s, 
including  35  planned  for  deployment  at  general  support  (now  intermediate  rear),  with  the  rest 
installed  at  Army  depots  (plus  one  at  Fort  Gordon,  Georgia,  for  technical  training)  and  contractors 
(primarily  for  TPS  development).  An  additional  46  were  planned  for  procurement  from  FY85  through 
FY88,  for  a  total  inventory  of  188.  These  numbers  are  under  review,  however,  because  the 
AN/MSM-105  was  planned  for  use  at  intermediate-forward  maintenance  to  support  some  systems 
(e  g.,  DIVAD  and  AH-64  division  level),  but  such  usage  is  clearly  not  suitable.  Instead,  the 
intermediate  forward  test  equipment  (IFTE),  now  in  development,  may  be  used  at  that  location. 
Because  some  of  the  equipment  has  become  obsolete  in  the  meantime  (e  g.,  computers  and  periph¬ 
erals),  several  engineering  changes  have  been  approved  over  the  years.  The  Army  plans  to  rely  on 
the  AN/USM-410(V)  through  the  late  I990’s. 

The  Army  has  had  additional  problems  with  its  standardization  plans  for  ATE  at  the 
direct  support  level,  now  called  intermediate  forward.  The  Army’s  need  at  this  maintenance  level  is 
completely  different  from  that  of  the  Air  Force  or  Navy.  Intermediate-forward  maintenance  units 
must  support  a  wide  variety  of  different  weapons  and  communications  systems  in  adverse  terrain 
under  highly  mobile  conditions.  Without  proliferating  peculiar  ATE,  this  maintenance  requirement 
translates  into  a  standard,  modular,  bus-structured  architecture  that  can  be  tailored  or  reconfigured 
to  the  supported  commodity  such  that  each  ATE  system  will  be  compact  enough  to  be  housed  in  a 
truck-mounted  shelter  (2-^-  to  5-ton  tactical  truck)  with  enough  space  left  for  stocking  interface 
devices  and  test  program  tapes  (needed  for  testing  the  supported  LRUs)  and  SRUs  (needed  for  LRU 
repairs).  This  summarizes  the  operational  requirement  for  the  "base  station  test  facility”  (BSTF), 
which  is  one  of  the  products  of  the  IFTE  program  [The  proposed  ROC,  April  1980,  includes  additional 
required  capabilities  such  as  ( 1)  screening  of  SRUs  in  addition  to  testing  of  LRUs,  (2)  meeting  new 
test  requirements  through  open-ended  extensions  of  test  capabilities  with  minimum  development 
effort,  (3)  possessing  self-test  and  self-alignment  capabilities,  etc.  | 

The  IFTE  program  includes  three  more  products.  One  is  the  conceptual  development  of 
an  "electro-optical  test  facility.”  The  second  is  a  "contact  test  set”  (CTS),  which  is  a  portable  tester  for 


on-line  testing,  supplementing  the  weapons  system  built-in  test  and  facilitating  fault  isolation  to  the 
removable  LRU.  The  third  product  is  software,  including  system  software,  support  software,  and 
application  test  programs  for  the  planned  prototype  demonstrations.  The  first  phase  of  the  IFTE 
program  was  conducted  from  June  through  December  1983,  with  five  contractors  participating. 
Secretary  of  the  Army  review  prior  to  transitioning  to  full-scale  engineering  development  occurred  in 
December  1984.  A  full-scale  engineering  development  contract  was  awarded  in  September  1985.  The 
contractor  will  build  two  prototypes,  tailor  the  ATE  to  two  selected  weapons  systems,  demonstrate  the 
capabilities  of  the  ATE,  and  define  the  commodity-oriented  ATE  configurations.  Prototypes  are 
scheduled  to  be  available  for  evaluation  in  late  1988,  with  production  to  start  in  late  1989,  and  first 
unit  equipped  in  mid-1990.  One  of  the  unanswered  questions  at  the  present  time  is  to  what  extent  the 
equipment  of  the  "base  shop  test  station”  (i.e.,  the  actual  ATE  system  with  up  to  two  per  truck- 
mounted  BSTF)  will  be  commercial  or  militarized.  The  ROC  is  neutral  on  this  topic  (although  it  sets 
minimum  R&M  requirements),  while  the  Statement  of  Work  for  Phase  1  required  tradeoff  studies  on 
this  topic.  (In  contrast,  the  ROC  requires  the  CTS  to  be  fully  militarized.) 

The  past  uncertainty  and  delays  with  the  IFTE  program  have  encouraged  development 
and  fielding  of  a  variety  of  ATE  to  support  weapons  systems  fielded  since  the  late  1970’s  —  ATE  that 
may  not  be  compatible  with  the  future  IFTE.  For  example,  the  ATE  used  in  support  of  the  MI  tank 
and  the  M2/M3  Fighting  Vehicle  Systems  (FVS)  provides  a  good  illustration.  At  the  organizational 
level,  the  Ml  is  supported  by  the  simplified  test  equipment  (STE) -Ml,  later  modified  to  the 
STE-M1/FVS  to  provide  commonality  with  the  STE  for  the  M2/M3.  It  provides  46  test  routines  to  per¬ 
form  on-vehicle  checkout  and  fault  isolation  to  one  or  more  LRUs,  supplementing  the  limited  built  in 
test.  The  STE  is  packaged  in  seven  portable  cases,  five  containing  adaptors  and  cables  and  two  the 
test  equipment,  and  is  transported  by  truck.  To  correct  several  shortcomings  in  the  STE.  a  successor 
program,  the  Simplified  Test  Equipment-Expandable  (STE-X),  is  currently  in  development,  with  a 
full-scale  development  decision  expected  early  next  year  6  Clearly,  the  STE-X  and  the  CTS  have  the 
same  function  and  purpose,  but  the  two  programs  apparently  are  proceeding  in  isolation  of  each 
other  Similarly,  at  the  intermediate-forward  maintenance  level,  the  Ml  and  FVS  are  supported  by 
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the  Direct  Support  Electrical  System  Test  Set  (DSESTS),  which  is  installed  in  a  truck-mounted 
shelter  and  provides  LRU  testing  and  repair  of  12  LRUs  from  the  Ml  and  13  LRUs  from  the  M2/M3. 
The  DSESTS  is  fully  militarized  equipment  (MIL-T-28800C  Type  1,  Class  3,  Style  A).  As  of 
September  1984,  the  Tank  Automotive  Command  had  procured  approximately  600  STEs  and 
300  DSESTSs.  Compatibility  between  DSESTSs  and  the  BSTF  apparently  has  not  been  specified  in 
any  plan. 

The  Army’s  policy  on  standardizing  intermediate-level  ATE,  promulgated  by  message 
dated  6  June  1984  and  effective  as  of  1  July  1984,  is  as  follows: 


IFTE-BSTF/CTS  will  be  the  standard  automatic  testers  for  intermediate  electronic 
maintenance  applications  for  all  systems  scheduled  for  initial  fielding  after  first 
quarter  FY89. 


Systems  entering  production  or  full-scale  development  after  second  quarter  FY86 
must  use  IFTE  components  where  ATE  is  required. 


Programs  that  currently  rely  on  the  use  of  other  ATE  systems  and  will  remain  in  use 


after  FY92  must  conduct  economic  analysis  to  determine  the  advisability  of  convert¬ 
ing  their  TPSs  to  IFTE.  No  product  improvements  to  prolong  the  life  of  nonstandard 
ATE  will  be  authorized  after  FY87  without  an  economic  analysis  and  the  approval  of 
PM.TMDE. 


Development  of  unit  level  ATE  will  not  be  initiated  without  appropriate  approval  by 
PM,  TMDE.  The  STE-X/CTS  will  first  be  considered  as  the  objective  ATE  solution  to 
test  requirements,  to  include  development  of  expansion  modules.  Specific  plans  to 
use  a  system  other  than  STE-X/CTS  must  be  identified  to  PM,  TMDE  Systems 
fielded  after  first  quarter  FY88  or  entering  development  after  third  quarter  FY84 
must  consider  the  STE-X/CTS  as  their  prime  candidate  for  unit  level  ATE. 


Waivers  of  the  standard  ATE  policy  must  be  requested  in  accordance  with  Army 
Regulation  740-43  to  obtain  concurrence  of  PM,  TMDE. 


All  configurations  of  the  AN/USM-410  or  OQ-290  used  forward  of  the  echelon  above 
corps  level  will  be  programmed  for  replacement  by  IFTE. 


In  summary,  the  Army  only  recently  decided  on  a  clear  direction  for  its  ATE  programs. 
Apparently,  that  direction  was  needed  in  light  of  recent  problems,  such  as  the  fielding  at 


6The  practical  problems  associated  with  the  use  of  STE  in  the  field  are  described  in  Frans 
Nauta,  Fix-Forward:  A  Comparison  of  the  Army’s  Requirements  and  Capabilities  for  Forward 
Support  Maintenance.  Task  ML104  (Bethesda,  Maryland:  Logistics  Management  Institute,  April 
1983)  (AD  A133954) 
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intermediate  (rear)  maintenance  of  a  standard  ATE  that  was  already  obsolete,7  and  the  lack  of  stand¬ 
ard  ATE  at  intermediate  (forward)  maintenance  until  at  least  the  early  1990’s.  If  the  IFTE  program 
is  successful,  it  will  solve  a  large  part  of  the  Army’s  present  ATE  problems. 


Marine  Corps 

Because  ATE  requirements  are  driven  by  the  supported  weapons  systems,  it  is  conven¬ 
ient  to  distinguish  between  Marine  Corps  aviation  and  ground  equipment.  The  latter  have  had  little 
or  no  need  for  field-level  ATE  in  the  past,  but  the  technology  of  new  weapons  systems  being  intro¬ 
duced  is  beginning  to  impose  such  a  need.  The  only  item  of  standard  ATE  in  the  inventory  is  the 
AN/USM-465  digital  card  tester  (or  its  functional  equivalent,  HMC-193,  the  Hughes  Aircraft  Corpo¬ 
ration’s  version  of  the  GENRAD-2225),  which  is  also  used  by  the  Army  and  surface  N'avy.  Compared 
to  the  hundreds  of  TPSs  fielded  in  the  Army,  the  Marine  Corps  ground  has  less  than  10,  but  over 
100  are  under  development. 

In  contrast,  Marine  Corps  aviation  has  more  experience  with  ATE.  Materiel  acquisition 
for  aviation  {aircraft,  support  equipment,  provisioning)  is  a  Navy  responsibility.  As  a  result,  Marine 
Corps  aircraft  that  are  also  in  the  Navy  inventory  are  supported  the  same  way  For  example,  the 
Marine  Corps  supports  the  F/A-18  with  the  same  type  of  maintenance  organization  as  in  the  Navy, 
with  the  identical  large-scale  ATE  at  the  intermediate  level.  For  the  F/A-18,  this  includes 
three  members  of  the  Navy’s  standard  ATE  family  and  one  peculiar  test  system  for  the  F/A-18  radar 
system  ( AN/APG-65)  WRAs: 

•  IML’TS  II  (AN/USM-608).  A  special  purpose  hybrid  tester  for  inertial  navigation 
system  WRAs,  which  currently  supports  both  the  AN/ASN  92  (not  on  the  F-18)  and 
the  AN/ASN- 130  aboard  the  F/A-18  This  tester  is  of  rack  and-stack  design,  two 
racks  wide. 


7For  example,  in  a  letter  report  to  AMC,  the  U  S.  General  Accounting  Office  (GAO)  quotes  a 
December  1983  message  from  the  Army’s  European  Logistics  Assistance  Office  to  AMC  Head¬ 
quarters,  the  message  concludes  that  "further  fielding  of  the  EQUATE  (AN/MSM-105)  within  the 
U  S.  Army  Europe  will  offer  no  improved  readiness  or  any  increased  ability  to  go  to  war  ”  The 
European  office  requested  that  fielding  be  halted  in  favor  of  leasing  state-of-the-art  ATE.  See:  U  S 
General  Accounting  Office,  "GAO  Concerns  Relating  To  Additional  EQUATE  Procurements  and 
Improvements,”  GAO  Flash  Report  No.  NSIAD-84-152  (Washington,  D  C  U  S.  General  Account¬ 
ing  Office,  17  August  1984). 


•  Automatic  Test  System  f AN/USM-470(V)11:  A  general-purpose  hybrid  tester  for 
avionics  WRAs.  Also  referred  to  as  the  Mini-VAST  (Versatile  Avionics  Shop  Tester), 
this  ATE  combines  some  second-generation  stimulus  building  blocks  from  the  VAST 
with  commercial  third-generation  measurement  instruments.  The  computer  and 
peripherals  are  commercial  off-the-shelf  equipment.  The  system  comprises  six 
racks. 

•  Hybrid  Test  System  (AN/USM-484);  A  general-purpose  hybrid  tester  for  avionics 
shop  replaceable  assemblies.  It  has  some  commonality  with  the  Navy’s  other 
standard  testers,  and  uses  primarily  commercial  equipment,  with  the  same 
computer  and  peripherals  as  used  in  the  automatic  test  system.  It  is  installed  in  four 
racks. 

•  RTS  (Radar  Test  Station):  Special-purpose  tester  for  radar  WRAs,  installed  in 
5^  racks  of  equipment.  Most  of  the  equipment  (about  75  percent)  is  commercial  off- 
the-shelf,  the  rest  is  specially  designed  for  this  ATE. 

Although  the  AV-8B  Harrier  II  program  is  managed  by  the  Navy,  it  is  being  acquired  to 
meet  a  peculiar  Marine  Corps  requirement.  The  ATE  for  intermediate-level  maintenance  of  the 
AV-8B  was  developed  to  meet  the  needs  for  forward  deployment.  As  a  result,  it  consists  of  four  small 
testers  (each  composed  of  three  fully  militarized  boxes,  two  common  among  the  four  testers  and  one 
peculiar  to  the  testing  application:  controls,  displays,  navigation,  and  armament)  that  provide  the 
full  capability  of  WRA  testing  at  forward  sites.  Known  as  the  "expeditionary  test  set,”  it  meets  the 
operational  requirements  of  minimum  size  and  weight,  mobility,  and  resistance  against  rough  envi¬ 
ronmental  conditions.8  It  represents  the  first  field-level  ATE  in  the  DoD  inventory  that  was 
developed  with  particular  emphasis  on  the  operational  environment  in  wartime.  Prototype  demon¬ 
strations  were  held  in  1982,  and  first  delivery  of  the  production  version  will  begin  in  late  1985. 

To  deal  with  the  increasing  need  for  ATE  in  nonaviation  weapons  system  support,  the 
Deputy  Chief  of  Staff  (Installations  and  Logistics)  chartered  a  working  group  in  1980  to  formulate 
Marine  Corps  policy  for  the  development,  acquisition,  utilization,  and  management  of  ATE  The  out¬ 
come  of  that  effort  was  the  establishment  of  the  Automatic  Test  Services  Unit  (ATSU)  within  the 
depot  maintenance  activity  at  Marine  Corps  Logistics  Base,  Albany,  Georgia,  and  the  issuance  of 
Marine  Corps  Order  10510.18,  "Policy  and  Responsibilities  for  Test,  Measurement,  and  Diagnostic 


8A  technical  description  may  be  found  in:  D  L  Williams  and  W  .J  Austin,  "The  Expedi 
tionary  Test  Set —  A  Fresh  Approach  To  Automatic  Testing,”  Proceedings  AUTOTESTCON  '83 
(New  York:  The  Institute  of  Electrical  and  Electronics  Engineers,  1983),  pp  186-191. 


Equipment  (TMDE)”  (current  version  A,  dated  2  February  1982).  That  policy  emphasizes  the  prefer¬ 
ence  for  general-purpose  TMDE,  but  points  out  that  TMDE  selection  should  not  be  based  on  standard¬ 
ization  alone;  other  factors  include  cost  effectiveness,  environmental  requirements,  state-of-the-art, 
and  future  measurement  requirements.  As  a  general  rule,  non-Fleet  Marine  Force  units  (i.e.,  the 
supporting  establishment  in  the  continental  United  States)  will  use  commercial  equipment,  while 
Fleet  Marine  Force  units  may  use  modified  commercial  or  militarized  equipment  if  commercial  equip¬ 
ment  is  inadequate.  The  Marine  Corps  Order  also  established  a  standard  ATE  program  which 
requires  that  hardware  and  software  of  planned  ATE  be  compatible  with  the  specific  parameters  and 
characteristics  of  the  standard  ATE.  To  stem  proliferation  of  ATE,  all  procurements  or  modifications 
of  ATE  need  to  be  approved  by  the  Commandant  of  the  Marine  Corps  (Material  Acquisition  Support 
Branch)  The  ATSU  was  established  to  provide  the  technical  and  engineering  services  in  support  of 
the  standard  ATE  program,  including 

•  Development  of  TPSs 

•  Certification  and  approval  of  all  TPSs  prior  to  fielding 

•  Configuration  control  of  all  TPSs  and  ATE 

•  Technical  assistance  to  Headquarters  Marine  Corps  and  ATE  users 

•  Advice  on  ATE  selection  to  meet  specific  support  requirements. 

Since  its  inception,  ATSU  has  taken  the  lead  in  developing  the  concept  for  a  Marine 
Corps  Automatic  Test  Equipment  System  (MCATES)  and  is  currently  actively  engaged  in  imple¬ 
menting  it.  The  ROC  for  MCATES  was  approved  in  October  1984.  MCATES  comprises  two  separate 
products:  (1)  a  TPS  Development  and  Management  System  for  installation  at  ATSU,  and  (2)  common 
ATE  stations  for  fielding  at  both  depot-  and  intermediate-level  maintenance  organizations  in  the 
Fleet  Marine  Force. 

The  TPS  Development  and  Management  System  is  designed  to  support  development, 
verification,  production,  distribution,  and  configuration  management  of  test  programs  It  consists  of 
a  Digital  Equipment  Corporation  VAX- 11/780  computer  with  a  series  of  micro- VAX  work  stations 
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connected  through  a  network.  Test  programs  will  be  developed  in  the  DoD-approved  Institute  of  Elec¬ 
trical  and  Electronics  Engineers,  Inc.  (IEEE)  Standard  716,  ''Common/Abbreviated  Test  Language 
for  All  Systems  (C/ATLAS)  Language.”  The  Marine  Corps  will  use  the  Navy-developed  Hierarchical 
Integrated  Test  Simulator  as  its  standard  digital  automatic  test  program  generator. 

The  MCATES  test  stations  are  fully  commercial  They  use  a  standard  bus  architecture 
(IEEE  Standard  488-1978,  "Standard  Digital  Interface  for  Programmable  Instrumentation”)  and  will 
be  installed  in  standard  Marine  Corps  shelters  and  vans  ATSU  has  selected  the  68000  micro¬ 
computer  with  UNIX  operating  system  software  as  the  standard  instrument  controller;  standard 
peripherals  include  cathode  ray  tube,  disk  drive,  and  dot  matrix  printer  ATSU  is  currently  in  the 
process  of  selecting  commercial,  general-purpose  ETE  compatible  with  the  IEEE  488-1978  bus.  For 
the  test  head,  ATSU  has  established  the  following  characteristics:  direct  memory  access  with  data 
transfer  at  500K  bytes/second;  drivers/sensors  with  a  dual  threshold  over  a  range  of  -28  to 
+  28  volts;  10  MHz  test  rate;  and  an  interface  with  748  pins  with  flexible  switching,  allowing  any 
individual  pin  to  provide  stimulus  or  measurement.  The  test  station  is  similar  to  the  commercial 
ATE  found  in  industry. 

Among  the  essential  characteristics  specified  by  the  ROC  are  the  following:  capability  of 
being  used  23.5  hours/day  with  30  minutes  for  scheduled  maintenance;  self-test  capability  to  fault 
isolate  to  a  single  card  or  module  90  percent  of  the  time  and  no  more  than  three  modules  the  remain¬ 
ing  10  percent  of  the  time;  and  resistance  to  shock  and  vibrations  within  the  shelter  or  van  in  which 
the  test  station  is  installed.  Importantly,  the  ROC  anticipates  obsolescence  on  a  5-year  cycle  with  the 
need  to  upgrade  the  instrument  controller  and  test  head  elements  under  a  preplanned  product 
improvement  program. 

The  first  MCATES  test  station  is  planned  for  completion  in  1985  with  full  operational 
capability  in  FY88.  The  inventory  objective  is  190  stations,  with  173  programmed  to  be  procured 
through  FY90,  at  a  total  procurement  cost  of  $20.6  million  (zero  development  cost).  The  15-year  life 
cycle  cost  for  these  units  and  ATSU’s  operation  of  the  TPS  Development  and  Management  System  for 
test  program  updates  (not  initial  development)  is  estimated  at  $57.4  million  (FY85  dollars) 


Observations 


The  ATE  programs  of  the  Military  Services  are  designed  to  meet  specific  requirements. 
As  a  consequence,  efforts  to  achieve  interservice  standardization  will  not  be  fruitful,  primarily 
because  most  of  the  programs  are  too  far  along  to  be  susceptible  to  major  change.  Moreover,  the  dif¬ 
ferences  in  operational  environment  and  mission  requirements  of  the  ATE  are  real.  The  Air  Force’s 
program  is  designed  to  control  costs,  the  Navy’s  to  maximize  throughput  capacity  within  a  con¬ 
strained  space,  the  Army’s  to  meet  mobility  and  diverse  workload  requirements  at  the  intermediate 
(forward)  level,  and  the  Marine  Corps’  (for  ground  equipment  support)  to  minimize  cost.  Because  of 
"vertical  commonality”  requirements  (to  ensure  compatibility  of  test  results),  efforts  to  standardize  at 
the  depot  level  (where  operational  environment  and  mission  requirements  are  identical)  would  prob¬ 
ably  yield  few  benefits. 

INVENTORY  MANAGEMENT 

Inventory  management  of  ETE  includes  such  actions  as:  monitoring  the  inventory  of  common 
ETE,  distributing  common  ETE  to  where  it  is  needed,  replacing  items  that  are  lost  or  disposed  of,  pro¬ 
visioning  spare  parts  for  items  that  are  org  lically  repaired,  monitoring  technical  obsolescence  and/ 
or  support  obsolescence  (unavailability  of  parts)  to  initiate  modernization  replacement,  and  mon¬ 
itoring  the  use  of  common  ETE  to  identify  shortcomings  in  fielded  items  and  initiate  corrective 
actions.  Many  of  these  functions,  however,  are  not  being  performed  or  they  are  being  only  partially 
performed  because:  the  function  was  not  included  in  the  organization's  formal  responsibilities;  the 
resources  (personnel,  money,  and  time)  are  not  available;  and/or  the  information  needed  to  carry  out 
the  function  is  not  available.  In  the  balance  of  this  section,  we  examine  several  aspects  of  inventory 
management  of  ETE  in  more  detail 

Management  Systems 

To  manage  a  large  inventory  of  equipment,  an  automated  data  base  and,  preferably,  a 
management  information  system  (MIS)  that  interfaces  with  the  data  base  are  required  None  of  the 
Military  Services  has  a  complete  and  accurate  data  base  for  common  ETE.  The  implications  are  that 


overages/shortages  in  test  equipment  cannot  be  identified,  pending  obsolescence  cannot  be  antici¬ 
pated,  and  contingency  plans  for  meeting  shortfalls  cannot  be  maintained. 

The  Army  is  developing  an  automated  data  base.  A  test  equipment  MIS  development 
effort  is  currently  underway  at  CTA,  with  a  first  version  scheduled  to  be  available  in  September  1985. 
However,  the  data  base  will  not  be  fully  loaded  for  several  years. 

The  Navy  developed  its  first  automated  data  base  for  GPETE  in  August  1984  by  taking 
an  extract  off  the  MEASURE  (Metrology  Automated  System  for  Uniform  Recall  and  Reporting)  data 
base.  NAVELEX  believes  that  data  base  contains  85  percent  of  its  GPETE  inventory.  It  does  not 
plan  to  develop  a  separate  MIS  for  GPETE. 

The  Air  Force  has  several  computer  systems  that  together  constitute  the  Air  Force 
Equipment  Management  System  The  Support  Equipment  Acquisition  and  Control  System 
(CodeC013)  contains  data  on  all  support  equipment,  including  ETE  and  ATE,  in  the  Air  Force’s 
inventory  That  system,  developed  at  Ogden  ALC,  Utah,  in  1975  and  operational  since  1979,  was 
transferred  to  AFLC  Headquarters  in  1982  and  is  designed  to  provide  a  central  means  for  collecting/ 
disseminating  data  on  support  equipment  availability  through  16  monthly  and  quarterly  reports  to 
ALC  item  and  system  program  managers,  major  commands,  and  Air  Staff.  Specific  procedures  and 
reporting  formats  are  described  in  Air  Force  Manual  67-1,  U  S.  Air  Force  Support  Manual.  Vol.  IV, 
Part  1,  Chapter  28,  March  1982  and  Vol  III,  Part  7,  August  1979  A  recent  audit,  however,  reported 
that  this  data  base  contains  many  errors  and  is  incomplete,  that  past  attempts  to  correct  the  deficien¬ 
cies  have  not  been  successful,  and  that  the  reports  generated  by  this  system  are  not  used  9  The 
long-term  solution,  according  to  AFLC,  is  to  upgrade  the  system  as  an  integral  part  of  the  Air  Force 
Equipment  Management  System  redesign  program.  Implementation  of  the  latter  effort  is  planned  for 
FY87 


9Air  Force  Audit  Agency,  "Review  of  the  Support  Equipment  Acquisition  and  Control 
System"  (Norton  Air  Force  Base,  California:  Headquarters  Air  Force  Audit  Agency,  lOJuly  1984) 
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The  shortcomings  in  that  system  have  forced  San  Antonio  ALC,  the  Air  Force’s  item 
manager  for  FSC  6625,  to  work  from  a  manual  ledger  system  to  manage  and  control  an  inventory 
estimated  at  800,000  items.  With  respect  to  ATE,  however,  the  San  Antonio  ALC  has  developed  the 
only  ATE  MIS  in  existence. 

Provisioning 

Most  of  the  ETE  inventory,  both  general-purpose  commercial  equipment  or  special- 
purpose  militarized  equipment,  is  repaired  in-house.  To  make  those  repairs,  spare  parts  must  be  pro¬ 
visioned  and  stocked.  Yet,  much  of  DoD’s  test  equipment  tends  to  become  unsupportable  within  a  few 
years  of  fielding  because  the  spare  parts  are  not  available.  The  cause  of  this  rapid  "support  obsoles¬ 
cence”  of  test  equipment  (i.e. ,  the  inability  to  provide  spare  parts  within  a  relatively  short  time  of 
procuring  the  end  item)  is  twofold.  One  is  the  lack  of  long-term  planning  of  spare  parts  requirements, 
and  the  other  is  the  reluctance  to  accept  substitute  parts  in  an  attempt  to  maintain  strict  configura¬ 
tion  control,  which  makes  item  management  easier. 

Commercial  test  equipment  vendors  follow  a  variety  of  procedures  in  providing  product 
support  after  production  of  the  end  items  is  discontinued.  The  following  describes  the  procedures  of 
one  of  the  larger  manufacturers,  Tektronix,  which  has  a  corporate  program  for  support  that  is  prob¬ 
ably  more  extensive  than  that  provided  by  many  other  vendors. 10  For  the  first  6  years  after  the  prod¬ 
uct  is  dropped  from  the  catalog,  the  company  provides  "full  support”;  i.e.,  replacement  parts  are  guar¬ 
anteed  to  be  available.  In  the  7th  through  9lh  year  after  phaseout,  the  company  provides  "limited  sup¬ 
port,”  which  means  maintenance  service  is  no  longer  company-wide,  but  limited  to  designated  service 
centers;  replacement  parts  are  stocked  according  to  demand  and  vendor  availability,  while  prices  will 
escalate  because  of  custom  manufacturing;  and  accessories  and  calibration  fixtures  may  no  longer  be 
available.  After  the  9th  year,  the  product  is  considered  "obsolete”;  i.e.,  technical  support  is  only  avail¬ 
able  on  a  "best  ellort”  basis,  product-specific  parts  are  no  longer  available,  technical  information  is 

l0The  source  of  the  information  is  a  paper  presented  by  Walter  Karsted,  "Long-Term  Product 
Support  at  Tektronix,”  Proceedings  of  the  Electronic  Test  Equipment  Division’s  Annual  Program 
Review  (Arlington,  Virginia:  American  Defense  Preparedness  Association,  May  1984) 
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still  available  for  another  6  years,  and  manufacturing  rights  may  be  made  available  to  other  parties 
on  a  case-by-case  basis.  Exceptions  to  this  corporate  product  support  program  include  some  original 
equipment  manufacturer  products,  probes,  and  accessories;  suppliers  going  out  of  business;  and 
"special  products.”  Tektronix  informs  its  customers  about  the  support  situation  for  its  products  and 
defines  the  last  year  of  support  in  advance  to  prepare  the  customer  for  replacement  or  parts  buy-out 
decisions.  While  the  postproduction  support  offered  by  Tektronix  may  be  better  than  that  of  some 
other  test  equipment  manufacturers,  advance  customer  notification  of  product  phaseout  and  support 
termination  are  standard  industry  practices. 

Because  of  competitive  pressures  in  the  commercial  ETE  marketplace  and  rapid  tech¬ 
nology  advances,  the  time  a  particular  product  is  in  production  is  relatively  short,  and  it  is  becoming 
shorter.  It  may  take  only  2  or  3  years  for  a  product  to  be  phased  out  and  replaced  by  another  model 
offering  competitive  advantages  (more  capability  or  lower  price).  Furthermore,  with  the  exception  of 
some  special  products  (e  g.,  high  technology  areas  such  as  microwave  test  equipment),  DoD  is  but  a 
small  customer  overall.  The  notion  that  DoD  can  influence  significantly  the  commercial  ETE  indus¬ 
try  is  unrealistic. 

The  spare  parts  problem  for  special,  militarized  test  equipment  is  even  more  serious 
than  it  is  for  common,  commercial  test  equipment.  After  the  last  end  item  has  been  produced  and 
delivered  under  a  procurement  contract,  manufacturers  generally  do  not  keep  a  production  line  open 
to  provide  spare  parts  to  the  DoD.  While  initial  provisioning  parts  are  normally  included  in  the  end 
item  procurement  contract,  the  quantities  of  those  parts  are  more  often  based  on  available  budget 
than  on  an  engineering  assessment  Buying  replenishment  parts  later  usually  is  very  expensive  and, 
in  most  cases,  not  worth  the  effort  to  the  manufacturer  because  they  would  have  to  be  produced  at  the 
expense  of  current  production,  which  is  needed  to  meet  current  market  demands. 

The  solution  to  the  parts  supply  problem  of  both  classes  of  test  equipment  is  straight¬ 
forward  Item  managers  need  to  project  the  long-term  parts  requirements  for  each  end  item  when 
they  buy  it,  update  those  projections  based  on  experience  data  from  the  field,  and  procure  the  required 
parts  in  multiyear  contracts.  They  also  need  to  act  upon  product  support  notices  received  from 


industry.  If  cost  effective,  they  should  continue  supporting  fielded  equipment  with  substitute  parts 
(interchangeable  by  form,  fit,  and  function,  or  parts  that  are  upwards  compatible)  under  appropriate 
configuration  management  (not  control)  procedures;  otherwise,  they  should  phase  out  the  fielded 
equipment  for  which  no  parts  are  available.  Additionally,  the  Military  Services  need  to  elevate  the 
funding  priority  for  test  equipment  spare  parts  above  that  for  new  test  equipment. 

Modernization 

The  Marine  Corps  estimates  that  more  than  one-half  of  its  principal  items  of  common 
test  equipment  (350  line  items,  excluding  small  value  items  and  ancillary  items)  is  8  years  or  older. 
25  percent  is  12  years  or  older.  It  has  no  specific  procedure  for  identifying  obsolete  items,  nor  does  it 
have  any  modernization  plans  or  programs. 

NAVELEX  estimates  that  13  percent  of  its  GPETE  line  items  (which  amount  to  about 
30,000  line  items  with  a  550,000  item  count)  is  obsolete  because  spare  parts  are  not  available 
NAVELEX  has  been  submitting  budget  requests  for  replacement  of  obsolete  items,  but  these  requests 
have  not  been  funded  by  the  Navy.  It  does  not  have  a  separate  line  item  for  modernization  of  fielded, 
obsolete  equipment,  those  requirements  are  commingled  with  procurement  of  new  common  ETE  in  a 
single  budget  line.  Since  GPETE  is  a  "stock  funded”  item  in  the  Navy,  Naval  Supply  Systems 
Command  is  responsible  for  replacing  equipment  that  is  damaged  beyond  repair  or  otherwise  dis¬ 
posed  of  and  also  for  funding  depot-level  repairs  to  maintain  a  ready-for-issue  inventory  in  the  supply 
system. 11  (As  far  as  the  Fleet  is  concerned,  the  stock  funding  of  GPETE  means  that  a  unit  can  obtain 
a  replacement  for  a  damaged  item  only  by  submitting  to  SPCC  a  funded  requisition  citing  the  Type 
Commander’s  operating  budget  or,  in  the  case  of  aviation  units,  to  the  Aviation  Supply  Office)  It 
appears  that  such  a  fine  distinction  between  an  item  that  is  unrepairable  because  of  damage  (with  a 
repair  limit  set  at  75  percent  of  the  unit  price1^)  and  an  item  that  is  unrepairable  because  repair  parts 

UNAVELEX  Instruction  5450  229,  "Policies,  Procedures,  and  Responsibilities  for  Navy 
General  Purpose  Electronic  Test  Equipment  (GPETE),”  (Washington,  D  C  Department  of  the 
Navy,  24  June  1974) 

12Marv  B  Acton  (Deputy  for  Management  and  Analysis),  Repair  Strategy  Assessment. 
Part  I  —  Depot  Maintenance  (Alexandria,  Virginia  Headquarters  Army  Materiel  Command, 
January  1985) 


are  not  available  would  create  problems  for  the  Navy  in  budgeting  for  modernization  of  obsolete  test 
equipment 

San  Antonio  ALC  attempts  to  review  at.  A  its  FSC  6625  line  items  every  5  to  7  years 
Those  reviews  identify  items  that  are  or  soon  will  become  unsupportable  and  result  in  budget  require¬ 
ments  for  replacements  In  view  of  the  magnitude  of  the  inventory  154,000  NSNs),  this  approach  has 
serious  practical  limitations 

The  Army  is  the  only  Military  Service  that  has  a  formal,  separately  funded  and  fenced, 
common  TMDE  modernization  program  (TEMOD) 

The  total  Army  inventory  of  TMDE  is  estimated  to  consist  of  58,000  separate  makes  and 
models  (NSNs),  with  18,000  NSNs  in  FSC  6625  1;j  The  high-densitv  line  items  are  only  4,400  (all 
FSCs),  including  2,500  in  FSC  6625  The  TEMOD  program  is  focused  on  the  high-density  makes  and 
models  of  TMDE.  It  was  initially  limited  to  FSC  6625,  but  the  master  plan,  completed  in 
September  1983,  extended  the  program  to  all  FSCs  in  three  phases  (see  Table  2-1)  While  the  second 
phase  started  in  FY85,  the  first  phase  is  still  in  progress  Priorities  are  determined  in  coordination 
with  the  U  S.  Army  Training  and  Doctrine  Command  (TRADOC),  which  represents  the  users,  and  the 
first  such  meeting  took  place  in  May  1984  The  actual  management  of  modernization  in  classes  other 
than  FSC  6625  is  delegated  by  the  PM,  TEMOD  to  the  major  subordinate  command  that  is  the  item 
manager  for  the  equipment  concerned  (such  as  the  Missile  Command  for  FSC  4935). 

The  1981  funding  profile  was  $37  million  per  year  over  a  5-year  period  The  program, 
however,  has  not  been  fully  funded.  After  transfer  to  the  PM,  TEMOD,  the  funding  was  increased  to 
an  average  $30  million  annual  level  for  FY84  through  FY86,  with  an  additional  $10  million  in  the 
outyears.  Instead  of  representing  a  "one-shot”  program,  as  originally  conceived,  the  Army  has 
decided  to  replicate  the  program  in  the  future  on  a  7-  to  10-year  cycle. 

■^General  Richard  H.  Thompson,  Luncheon  Meeting  Speech,  National  Security  Industrial 
Association  1984  National  Conference  on  "Supporting  Weapon  System  Technology  Through  the 
I990’s”  ( Denver,  Colorado:  16  August  1984) 


TABLE  2-1  ARMYTMDE  MODERN  IZATION  PROGRAM 


PHASE 

START 

TMDE 

NUMBER 
OF  FSCs 

IDENTIFICA¬ 
TION  OF  FSCs 

NUMBER OF 
LINE  ITEMS 

PERCENTAGE 

OF  PROGRAM 

I 

FY81 

Common  ETE 

1 

6625 

2,543 

57.6 

II 

FY85 

Fire  control, 
missile,  aviation 

6 

4920,4931. 

4935,5985, 

6130.6630 

935 

21.2 

III 

FY89 

Other 

59 

— 

431 

9.8 

Excluded 

Other  DoD 

23 

— 

505 

11.4 

TOTAL 

89 

4,414 

100  0 

The  current  program  is  planned  to  reduce  the  2,543  high-densitv  line  items  of  common 
ETE  to  60  makes  and  models.  With  completion  of  the  Phase  III  modernizations,  the  3,909  high- 
density  line  items  of  all  TMDE  will  have  been  reduced  to  approximately  600  makes  and  models. 

In  summary,  the  Military  Services  have  not  provided  the  resources  nor  have  they 
established  the  policies  and  procedures  needed  for  effective  management  of  the  test  equipment  inven¬ 
tory.  Although  current  efforts  to  improve  management  information  will  bring  some  relief,  they  are 
not  sufficient.  In  addition,  both  the  Navy  and  Air  Force  need  to  adopt  a  formal  ETE  modernization 
program,  much  like  the  Army’s. 

PEACETIME  VERSUS  WARTIME  REQUIREMENTS 

Test  equipment  acquisition  and  inventory  management  in  each  of  the  Military  Services  is  sub¬ 
ject  to  budgetary  constraints  that  have  resulted  in  shortages  of  required  wartime  test  equipment 
The  precise  extent  of  these  shortages  is  unknown.  None  of  the  Military  Services  has  a  management 
system  in  place  to  (1)  monitor  wartime  test  equipment  requirements  versus  on  hand  inventory, 
<2)assess  the  impact  of  test  equipment  shortages-  on  maintenance  capabilities,  and  (3)  plan  emer¬ 
gency  procurements  of  test  equipment  needed  for  mobilization  or  in  support  of  wartime  contingencies 
The  implications  of  these  shortcomings  are  clear:  current  test  equipment  shortages  inhibit  meeting 
the  readiness  and  sustainability  objectives  stated  in  the  current  Defense  Guidance  While  it  may  not 
be  affordable  or  cost  effective  to  have  in  the  peacetime  inventory  all  the  test  equipment  required  in 
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wartime,  the  Military  Services  have  no  visibility  on  the  extent  of  the  shortfall  nor  on  the  contingency 
plans  needed  to  overcome  it. 


Army 

Although  most  of  the  Army’s  nondivisionai  intermediate  maintenance  capability  is 
provided  by  maintenance  units  in  the  Reserve  Components  (only  23  of  134  nondivisionai  main¬ 
tenance  companies  are  in  the  Active  Component),  most  of  those  units  do  not  have  the  tools  and  test 
equipment  required  to  perform  their  wartime  maintenance  mission.  For  example,  the  5th  Army, 
which  is  planned  for  early  mobilization  and  deployment  to  in  wartime,  includes  30  maintenance 
units,  26  of  which  are  in  the  Reserve  Component.  None  of  the  Reserve  Component  maintenance  units 
has  the  tools  and  test  equipment  to  carry  out  its  maintenance  mission.  In  1984,  the  U  S.  Army  Forces 
Command  (FORSCOM)  estimated  the  shortfall  in  test  equipment  authorized  by  Modified  Table  of 
Organization  and  Equipment  (MTOE)  at  approximately  $425  million  for  all  nondivisionai  mainte¬ 
nance  companies  in  the  Reserve  Component.  The  Army,  however,  is  not  funding  that  shortfall 

Furthermore,  many  of  the  tools  and  test  equipment  required  by  those  maintenance  units 
are  not  listed  in  their  MTOEs.  A  considerable  portion  may  be  hidden  under  a  "sets,  kits,  and  outfits” 
line  item  or  may  be  identified  only  in  the  Repair  Parts  and  Special  Tools  List  or  the  Maintenance 
Allocation  Chart  for  the  prime  equipment  that  the  units  will  be  supporting.  The  Army’s  extensive 
modernization,  currently  underway,  also  necessitated  modernization  of  the  special  tools  and  test 
equipment  possessed  by  Reserve  Component  maintenance  units.  The  Deputy  Chief  of  Staff  for  Logis¬ 
tics  (DCSLOG)  recently  initiated  a  program  to  purchase  special  tools,  test  sets,  and  other  mission- 
critical  items  not  authorized  in  unit  MTOEs.  This  program,  currently  known  as  Reserve  Component 
Transition  to  Modernization,  provides  for  incremental  funding  for  early  deploying  units  and  for 
selected  prime  equipments.  The  current  funding  level  is  at  $55  million  (FY85  through  FY90)  which 
covers  the  special  tools/test  sets  for  38  units  to  support  eight  new  weapons  systems.  This  program  is 
now  being  expanded  to  51  units  and  15  new  weapons  systems  Importantly,  the  Army  considers  the 
current  lack  of  such  non  MTOE  authorized  tools  and  test  equipment  for  nondivisionai  Reserve 
Component  units  to  be  potential  "war  stoppers  ” 
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Two  additional  observations  on  upgrading  the  tools  and  test  equipment  of  Reserve  Com¬ 


ponent  maintenance  units  appear  to  be  warranted.  One  is  that  the  Army  is  not  fielding  the  EQUATE 
with  the  Reserve  Component.  Yet,  that  equipment  will  be  the  main  item  of  test  equipment  that  light 
equipment  maintenance  companies  (intermediate  rear)  need  to  perform  their  mission.  Wartime 
operating  hours  for  much  of  the  Army’s  electronic  equipment  exhibits  a  steep  increase  over  peacetime 


operating  hours,  resulting  in  an  ATE  workload  that  is  a  large  multiple  of  that  in  peacetime.  Under 

r 

t 

the  planned  ATE  deployment,  much  of  that  workload  would  be  in  excess  of  ATE  throughput  capacity,  • 

necessitating  evacuation  of  the  excess  workload  to  continental  U  S.  depots  or  contractors  plus 
increased  forward  stockage  of  spares.  Concern  about  this  wartime  gap  in  ATE  capacity  was  first 
expressed  in  a  1979  study  conducted  by  the  Deputy  Chief  of  Staff  for  Research,  Development,  and 
Acquisition  and  forwarded  to  the  Commander,  AMC  14  The  Army  has  not  yet  resolved  this  issue. 

The  second  observation  is  that  the  MTOE  authorization  of  standard  test  equipment  may 
be  short  of  wartime  requirements.  Among  maintenance  supervisors  there  is  a  consensus  that  the  cur¬ 
rent  authorization  of  oneSTE-Ml  per  maintenance  company  is  insufficient  in  wartime  and  should  at 
least  be  doubled.  (Because  the  STE-M1  has  some  shortcomings  itself,  it  may  be  cost  effective  to  defer 
increasing  the  density  of  this  equipment  until  the  product-improved  version,  STE-X,  is  fielded.) 

However,  the  TRADOC  does  not  have  the  analytical  tools  to  assess  wartime  TMDE  requirements  in 
preparing  Basis  of  Issue  Plans. 

In  sum,  the  Army  has  a  wartime  test  equipment  shortfall  of  unknown  billions  of  dollars 
With  the  recent  DCSLOG  initiative,  it  has  taken  a  first  step  toward  resolving  that  shortfall. 

Navy 


The  Navy's  shortage  of  manual  ETE  in  the  active  Fleet  is  illustrated  best  bv  the  need  for 
"cross-decking”  support  and  test  equipment  between  ships  that  have  returned  from  a  cruise  and  those 
preparing  for  deployment.  On  the  one  hand,  this  procedure  may  be  viewed  as  maximizing  the  use  of 


14Lieutenant  Colonel  Douglas  H  Barclay,  "An  Analysis  of  the  Army's  Automatic  Test  Equip¬ 
ment  Program”  (Washington,  DC.:  Office  of  the  Deputy  Chief  of  Staff  for  Research,  Development, 
and  Acquisition,  November  1979),  forwarded  by  letter  from  Lieutenant  General  Keith  to 
General  Guthrie,  27  November  1979 
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expensive  support  equipment  and  reducing  inventory  and  ownership  costs.  On  the  other  hand,  it 
shows  that  the  Navy’s  test  equipment  inventory  is  insufficient  to  support  the  peacetime  operating 
tempo,  when  only  one-third  of  the  Fleet  is  deployed  at  any  time.  In  wartime,  when  the  Navy  plans  to 
at  least  double  its  operating  tempo,  cross-decking  would  no  longer  fill  the  test  equipment  allowances 
of  all  deploying  ships. 

The  impact  of  ATE  throughput  constraints  on  wartime  sustainability  of  naval  air  opera¬ 
tions  has  been  examined  in  a  few  studies.  For  example,  a  recent  study  concludes  that  ATE  capacity 
aboard  aircraft  carriers  is  the  main  bottleneck  in  supporting  the  planned  S-3A  wartime  flying 
hours.15  Since  carriers  do  not  have  space  for  additional  ATE,  the  only  alternative  is  to  increase 
shipboard  stocks  of  WRAs  and  SRAs  and  possibly  take  other  actions  such  as  those  explored  by  the 
Defense  Resource  Management  Study.15  The  Navy,  however,  has  so  far  not  resolved  this  issue  and 
appears  to  rely  on  the  next  generation  of  ATE  (CASS)  to  solve  the  problem;  CASS,  however,  will  not 
be  operational  until  the  early  1990’s. 

Air  Force 

The  Air  Force  estimates  that  it  has  a  $1.5  billion  shortfall  in  support  and  test  equip¬ 
ment,  with  almost  80  percent  of  that  shortfall  in  test  equipment.17  It  acknowledges  that  this  shortfall 
would  have  an  impact  on  its  wartime  operations,  but  it  does  not  have  any  formal  plans  to  eliminate 
the  shortfall. 


ATE  constraints  on  Air  Force  wartime  operations  is  a  recurring  theme  in  several 
studies.  For  example,  a  well-documented  study  by  The  RAND  Corporation  clearly  demonstrates  that 
F- 15  operations  cannot  be  sustained  in  wartime  with  the  planned  support  resources.15  The  study 


15T  F.  Lippiatt,  et  al.,  Carrier  Based  Air  Logistics  Study:  Integrated  Summary. 
R-2853-Navy  (Santa  Monica,  California  The  RAND  Corporation,  .January  1982)  ( AD  Al  13289). 

15Donald  B.  Rice  (Study  Director),  Defense  Resource  Management  Study  (Washington.  D  C 
L’  S  Government  Printing  Office,  February  1979) 

17Major  General  Monroe  T.  Smith  (Study  Chairman),  Support  Equipment  Acquisition 
Review  Group  Final  Report  (Wright-Patterson  Air  Force  Base,  Ohio  Air  Force  Acquisition 
Logistics  Center, -July  1984). 

15J  R  Gebman,  et  al.,  Support  Improvements  for  F-15  Avionics.  Vol  2,  Analysis  (U). 
R-2591/2-1  AF  (Santa  Monica,  California;  The  RAND  Corporation,  March  1983) 


estimates  that  several  hundred  million  dollars  must  be  invested  in  additional  ATE  and  additional 
avionics  LRUs  in  war  readiness  spares  kits  to  sustain  F-15  wartime  flying  hours 
Observations 

Considerations  of  peacetime  cost  effectiveness  have  driven  the  Military  Services  to  lower 
their  test  equipment  inventories  below  the  levels  required  to  sustain  wartime  operations.  Given 
competing  budget  demands,  this  appioach  may  be  sound  from  the  management  perspective  as  long  as 
the  test  equipment  shortfall  is  not  forgotten  and  plans  are  developed  to  meet  that  shortfall  in  an 


emergency. 


3.  TEST  EQUIPMENT  SUPPORT  MANAGEMENT 


This  chapter  describes  how  the  Military  Services  are  organized  to  repair  and  calibrate  ETE. 
OVERVIEW 

The  Military  Services  have  different  approaches  to  ETE  support.  Army  policy  calls  for  each 
unit  to  perform  organizational-level,  mainly  preventive,  maintenance  on  assigned  ETE.  More  com¬ 
plex  repairs,  as  well  as  all  calibration  of  ETE,  are  to  be  performed  by  specifically  designated  calibra¬ 
tion  and  repair  units  (for  general-purpose  TMDE)  or  intermediate  maintenance  units  (for  special- 
purpose  TMDE).  The  Navy  assigns  more  ETE  repair  responsibility  to  owning  units  than  does  the 
Army.  However,  if  those  units  encounter  repairs  beyond  their  capability,  they  must  find  another 
organization  that  can  perform  the  repairs.  The  calibration  of  ETE  within  the  Navy  is  separate  from 
the  repair  function;  it  is  performed  by  a  variety  of  calibration  laboratories  and  activities.  The  Air 
Force’s  approach  to  ETE  support  is  quite  different  from  that  of  the  Army  and  Navy  in  that  it  assigns  a 
substantial  ETE  repair  and  calibration  capability  to  operating  units  at  the  base  level. 

These  differences  in  ETE  support  are  understandable  considering  the  different  missions  that 
they  have  been  assigned  and  the  different  environments  in  which  they  must  operate  The  Army,  for 
example,  does  not  find  it  practical  to  equip  combat  units  with  the  capability  to  perform  significant 
repairs  of  ETE  or  to  calibrate  it.  The  Navy  places  similar  restrictions  upon  its  operating  units 
primarily  because  of  space  limitations  aboard  ships  In  contrast,  the  Air  Force  operates  from  large 
bases  that  can  be  readily  provided  with  the  capability  to  repair  and  calibrate  much  of  its  ETE 

The  organizational  structure  and  procedures  for  test  equipment  calibration  and  repair  are  com 
plex.  The  following  sections  first  provide  general  information  on  each  Military  Service's  calibration 
system  and  test  equipment  support  policies  as  introduction  to  a  summary  description  of  calibration 
and  repair  organizations  and  procedures.  The  chapter  concludes  by  outlining  some  concerns  about 
wartime  planning  for  test  equipment  support  and  the  support  equipment  (calibration  standards) 


I 
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CALIBRATION  SYSTEM  ORGANIZATION 

Calibration  is  defined  in  MIL-STD-1309C,  "Definitions  of  Terms  for  Test,  Measurement,  and 
Diagnostic  Equipment,”  as  follows: 

The  comparison  of  a  measurement  system  or  device  of  unverified  accuracy 
to  a  measurement  system  or  device  of  known  and  greater  accuracy  to  detect 
or  correct  any  variation  from  required  performance  specifications  of  the 
unverified  measurement  system  or  device. 

i 

Each  of  the  Military  Services  has  a  metrology  and  calibration  (METCAL)  program  to  ensure 
that  test  equipment  used  in  weapons  systems  maintenance  is  accurate.  Those  programs  consist  of  a 
hierarchy  of  reference  and  transfer  standards  that  are  traceable  to  the  national  standards,  and  are 
maintained,  controlled,  and  available  to  the  calibration  activities  responsible  for  test  equipment 
calibration. 

The  national  standards  for  mass,  length,  temperature,  light,  and  current  are  maintained  by  the 
National  Bureau  of  Standards  (NBS),  while  those  for  precise  time  and  time  interval  are  maintained 

by  the  Naval  Observatory.  As  shown  in  Figure  3-1,  the  high-level  laboratories  within  the  Military 

I 

Services  are  similarly  structured  but  there  are  many  differences  at  the  lower  echelons  A  brief 
description  of  each  echelon  follows. 

Type  I  Laboratories 

The  primary  standards  laboratories  maintain  the  highest  measurement  standards 
They  support  the  Type  II  laboratories  by  providing  metrology  engineering  and  calibration  services  for 
their  reference  standards.  The  Army  and  Air  Force  each  have  one  primary  standards  laboratory, 
while  the  Navy  has  two.  (Officially,  the  Navy  Primary  Standards  Laboratory  East,  located  in 
Washington,  D  C.,  is  a  detachment  of  the  Navy  Primary  Standards  Laboratory  West  that  is  located  at 
the  Naval  Air  Rework  Facility  (NARF),  North  Island,  San  Diego,  California)  The  Army’s  laboratory 
is  managed  and  operated  by  the  U  S.  Army  TMDE  Support  Group  (L'SATSG);  the  two  Navy 
laboratories  by  the  Naval  Air  Systems  Command  (N'AVAIR)  through  the  Naval  Aviation  Logistics 
Center  and  NARF  North  Island;  and  the  Air  Force  laboratory  by  the  Aerospace  Guidance  and 
Metrology  Center  (AGMC),  which  reports  to  the  AFLC  The  Air  Force  Measurement  Standards 
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FIGURE  3-1.  DoD  METROLOGY  AND  CALIBRATION  SYSTEM 


Laboratory  at  Newark  Air  Force  Station,  Ohio,  maintains  approximately  215  Air  Force  standards 
that  are  directly  traceable  to  the  national  standards  The  exact  number  of  Army  and  Navy  standards 
is  unavailable. 

Type  II  Laboratories 

The  level  directly  below  the  primary  standards  laboratories  includes  a  wide  variety  of 
reference  standards  laboratories.  Most  of  those  laboratories  provide  calibration  services  for  the 
working  or  transfer  standards  at  lower  level  laboratories  in  a  geographical  area  or  at  special  repair 
activities;  some  provide  calibration  services  for  test  equipment  beyond  the  capability  of  lower- level 
calibration  activities.  (Although  we  distinguish  here  between  calibration  equipment  installed  at 
calibration  activities  and  test  equipment  used  by  maintenance  personnel,  both  are  categories  of  test 
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equipment.)  In  spite  of  their  differences,  all  Type  II  laboratories  obtain  calibration  services  for  their 
standards  from  the  Type  I  laboratory. 


Army.  The  Army’s  secondary  reference  standards  are  maintained  by  Area  Calibration 
Laboratories  (ACLs),  which  are  an  organizational  element  of  Area  Calibration  and  Repair  Centers 
(ACRCs);  and  Internal  Calibration  Laboratories  (ICLs),  which  are  attached  to  Internal  Calibration 
and  Repair  Centers  (ICRCs).  ACRCs  (and  the  associated  ACLs)  are  established  by  AMC  as  sub¬ 
ordinate  elements  of  USATSG  to  provide  single-source  calibration  and  repair  support  for  TMDE  in  a 
geographical  area.  The  Army  has  five  ACRCs  [one  in  the  continental  United  States  (CONUS),  three 
in  Europe,  and  one  in  Korea|.  Each  ACRC  is  a  military  unit  organized  by  a  Table  of  Organization  and 
Equipment,  with  a  civilian  augmentation  per  Table  of  Distribution  and  Allowances  (TDA).  ICRCs 
(and  associated  ICLs)  are  civilian  TDA  organizations,  established  by  AMC,  but  are  located  at  six 
Army  depots  in  CONUS.  Both  ACLs  and  ICLs  are  operated  as  fixed  facilities,  although  the  ACLs  are 
deployable  but  can  only  operate  in  environmentally  controlled,  semifixed  facilities 

Navy.  The  Navy’s  Type  II  laboratories  include  Navy  Standards  Laboratories  located  at 
three  NARFs  (Norfolk,  Virginia;  Pensacola,  Florida;  and  Alameda,  California)  and  Reference  Stan¬ 
dards  Laboratories  located  at  all  Naval  Shipyards  and  Ship  Repair  Facilities,  some  Naval  Weapons 
Stations,  and  the  Navy  Gage  and  Standards  Laboratory,  Pomona,  California  They  are  all  shore- 
based  facilities,  established  under  approval  authority  of  NAVELEX  [Test  and  Monitoring  Systems 
(TAMS)  Program  Office,  ELEX-08T],  and  operated  as  civilian  TDA  organizations  They  provide 
regional  metrology  engineering  and  calibration  services  for  standards  and  test  equipment  beyond  the 
capability  of  Type  III  calibration  facilities.  Management  and  control  of  the  NARFs  is  exercised  by  the 
Naval  Aviation  Logistics  Center. 

Air  Force.  The  Air  Force  has  four  different  categories  of  Type  II  Precision  Measurement 
Equipment  Laboratories  (PMELs)  that  maintain  base  reference  standards  for  the  calibration/ 
certification  of  working  standards  utilized  at  the  next  lower  level  or  for  metrology  engineering  and 
calibration  services  beyond  the  capability  or  reach  of  the  next  lower-level  laboratories.  The  PMELs 
are  owned  and  operated  by  the  major  commands,  but  they  are  established  subject  to  the  approval  of 


AGMC,  Only  one  Type  II  PMEL  is  authorized  at  any  base  or  installation.  The  four  categories  of 
PMELs  are: 

•  Type  IIA  PMELs.  This  category  includes  six  laboratories  supporting  the  ALCs 
(including  AGMC)  and  two  laboratories  supporting  overseas  areas  that  are  operated 
by  theater  support  commands.  For  example,  the  Type  IIA  PMEL  supporting 
U  S.  Air  Forces  in  Europe  is  located  at  Bitburg  Air  Base,  Federal  Republic  of 
Germany.  The  Type  IIA  PMEL  supporting  the  Pacific  Air  Forces  is  located  at  Clark 
Air  Base,  The  Philippines. 

•  Type  IIB  PMELs.  This  category  includes  the  base  PMELs  located  at  all  major  Air 
Force  bases.  The  Air  Force  has  92  of  these  PMELs  worldwide. 

•  Type  11C  and  IIP  PMELs.  These  two  categories  are  special  PMELs  supporting 
research,  development,  test,  and  evaluation  programs  conducted  by  AFSC  Type  IIC 
PMELs  or  tailored  to  a  specific  mission  (IID).  The  Air  Force  has  10  Type  IIC  PMELs 
and  1  Type  IID  PMEL. 


In  drawing  parallels  between  the  Military  Services,  the  Air  Force’s  Type  IIA  PMELs  are 
comparable  to  the  Army’s  ACLs/ICLs  and  the  Navy’s  Standards  Laboratories/Reference  Standards 
Laboratories,  while  the  Type  IIB/C/D  PMELs  have  more  limited  capabilities  (The  Type  IIB  and  IIC 
PMELs  receive  calibration  support  from  Type  IIA  PMELs.) 

Type  III  and  IV  Laboratories 

The  lowest  levels  in  the  METCAL  system  comprise  a  wide  range  of  calibration  laborato¬ 
ries  and  activities,  with  pronounced  differences  among  the  Military  Services.  They  include  laborato¬ 
ries  maintaining  "working  standards”  for  calibration  of  test  equipment  as  well  as  various  "user 
calibration”  activities.  Most  of  these  laboratories  and  activities  are  staffed  primarily  by  military 
personnel.  All  are  supported  by  Type  II  standards  laboratories 

Army  The  Army’s  secondary  transfer  standards  are  operated  by  Area  TMDE  Support 
Teams  (ATSTs).  Thirty-four  ATSTs  are  civilian-staffed,  fixed  shops  located  at  research,  develop¬ 
ment,  test,  and  evaluation  activities  in  CONUS  (and  one  in  Panama)  for  dedicated  TMDE  support. 


The  balance  of  the  ATSTs  (140)  are  mobile  teams  with  truck-mounted  shelters  housing  the 


calibration  standards  These  teams  include  61  that  are  staffed  by  civilians  and  distributed  among  the 
6CONUS-based  ICRCs  and  79  military  teams  distributed  among  the  5  ACRCs  (45  teams  in  Europe, 
17  teams  in  Korea,  and  17  teams  in  CONUS).  Most  of  the  CONUS  teams  are  deployable.  The 


overseas  ATSTs  either  operate  from  fixed  sites  or  as  mobile  calibration  units  providing  on-site 
calibration  of  the  test  equipment  of  supported  maintenance  units.  The  Army  discontinued  user  cali¬ 
bration  ("Level  C”  calibration)  in  1979  with  its  reorganization  and  consolidation  of  general-purpose 
TMDE  calibration  and  repair.  Special-purpose  TMDE,  however,  is  not  supported  by  the  ATSTs  but 
through  designated  maintenance  channels. 

Navy.  The  Navy’s  Type  III  calibration  laboratories,  both  ashore  and  afloat,  provide 
calibration  and  repair  of  lower-level  standards  and  of  test  equipment  beyond  the  capability  of  Type  IV 
calibration  activities.  The  shore-based  calibration  laboratories  include:  Local  Calibration 
Laboratories  (located  in  shops  at  Naval  Shipyards  and  Ship  Repair  Facilities),  Navy  Calibration  Lab¬ 
oratories,  and  Fleet  Calibration  Laboratories  at  shore  intermediate  maintenance  activities  (SIMAs) 
and  Aircraft  Intermediate  Maintenance  Departments.  The  afloat  calibration  laboratories  include 
Fleet  Electronic  Calibration  Laboratories  aboard  each  tender  and  repair  ship  to  provide  calibration  of 
Fleet  ETE;  Fleet  Mechanical  Calibration  Laboratories  aboard  Fleet  Ballistic  Missile  (FBM) 
submarine  tenders  to  provide  calibration  services  for  mechanical  TMDE  from  both  the  tender  and 
FBM  submarines;  and  Mechanical  Instrument  Repair  and  Calibration  Shops  aboard  specific  tenders 
(other  than  those  supporting  FBM  submarines)  and  repair  ships  to  provide  calibration/repair  services 
for  mechanical,  pressure,  and  temperature  measuring  instruments  installed  aboard  ships.  The 
Type  III  calibration  laboratories  also  include  two  Marine  Corps  calibration  facilities,  located  at  the 
two  Marine  Corps  depots.  These  two  facilities  are  supported  by  a  designated  Type  II  Standards 
Laboratory  through  periodic  exchange  of  calibrated  standards.  (Marine  Corps  aircraft  are  supported 
by  designated  Navy  Calibration  Laboratories.) 

The  Navy  formally  recognizes  a  Type  IV  level  of  calibration,  known  as  Fleet  Calibration 
Activities.  These  activities  are  an  organic  part  of  organizational-  and  intermediate-maintenance 
organizations  ashore  and  afloat.  They  do  not  provide  scheduled  calibration  support  to  activities  other 
than  their  own  organization.  They  are  staffed  by  specially  trained  military  personnel  and  used 
primarily  to  calibrate  specific  TMDE  that  requires  calibration  each  time  it  is  used  or  on  a  very 
frequent  basis,  such  as  daily  or  weekly  The  Marine  Corps  has  similar  types  of  activities 


Air  Force.  The  Air  Force  has  formally  designated  Type  III  and  Type  IV  PMELs  Its 
12  Type  III  PMELs  are  tailored  to  specific  missions  and  receive  calibration  support  from  a  Type  II 
PMEL;  they  are  not  authorized  at  an  installation  where  a  Type  II  PMEL  exists  Type  IV  PMELs  are 
designated  to  support  selected  weapons  systems  through  transportable  calibration  equipment  in  both 
fixed  and  deployed  locations.  They  are  supported  by  a  designated  Type  II  PMEL.  The  Air  Force  has 
14  Type  IV  PMELs,  including  5  installed  at  locations  overseas.  Additionally,  30  of  its  Type  II  PMELs 
possess  transportable  field  calibration  units  that  may  be  deployed  to  provide  on-site  calibration  sup¬ 
port.  The  Air  Force  also  has  approximately  100,000  items  of  TMDE  that  are  calibrated  by  the  user. 

Observations 

Table  3-1  shows  that  the  Military  Services  operate  almost  500  nondedicated  calibration 
laboratories.  [Excluded  from  that  table  are  hundreds  of  dedicated  calibration  activities,  i.e.,  activities 
with  a  dedicated  calibration  mission  such  as  those  at  research  and  development  facilities  and  most 
TypelV-level  activities.  For  example,  the  Navy  itself  has  about  480  standards  and  calibration 
laboratories.IType  II  and  Type  III),  most  of  which  do  not  show  up  in  this  table  ]  Several  past  attempts 
to  consolidate  those  facilities  have  not  been  successful  although  the  benefits  from  consolidation 
appear  to  be  substantial. 

CALIBRATION  AND  REPAIR  POLICIES 

The  Military  Services  have  issued  detailed  policies  for  the  management  and  implementation  of 
their  METCAL  programs.  In  some  respects,  these  policies  are  very  similar  For  example,  they  have 
similar  approaches  to  determining  and  monitoring  calibration  intervals  of  test  equipment  and  they 
use  similar  types  of  recall  systems  to  control  the  calibration  of  general-purpose  test  equipment.  Their 
policies  also  emphasize,  to  various  degrees,  the  use  of  interservice  support  agreerr  nts  (ISSAs)  to 
reduce  costs  as  long  as  military  effectiveness  is  not  impaired.  In  other  respects,  the  policies  are 
dissimilar,  particularly  in  the  areas  of  management  and  control  of  the  calibration  and  repair  of  test 
equipment  and  in  the  maintenance  concepts  for  test  equipment.  A  brief  description  of  these  issues 
( management  and  control,  maintenance  concepts,  and  calibration  intervals)  follows. 
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TABLE  3-1.  SUMMARY  OF  NONDEDICATBD  CALIBRATION  LABORATORIES 


TRANSFER 

AND  CALIBRATION 

TOTAL 

128  fixed, 

168 

30  TFCUs1 

59  fixed, 

107 

26  mobile 

34  fixed,  61  semifixed. 

186 

79  mobile 

2  fixed,  3  mobile, 

13 

8  transportable 

284  fixed/semifixed 

146  mobile/transportable 

474 

MILITARY  primary  SECONDARY 
SERVICE  bkimaky  REFERENCE 


Air  Force 

Navy 

Army 

Marine  Corps 


TOTAL 


•Transportable  field  calibration  units. 

NOTE:  The  allocation  by  level  for  Air  Force  and  Navy  is  based  upon  responsibilities  (e  g  ,  the 
Air  Foi  ’s  Type  IIA  and  II D  laboratories  are  shown  as  secondary  reference  laboratories,  while 
Type  IIB,  IIC,  III,  and  IV  are  shown  as  transfer  and  calibration  laboratories) 

SOURCE  T  O.  00-20-14,  "Air  Force  Metrology  and  Calibration  Program,”  15  May  1982. 

Management  and  Control 

Army  The  Army’s  principal  calibration  policy  is  contained  in  Army  Regulation  750-25, 
"Army  Test,  Measurement,  and  Diagnostic  Equipment  Calibration  and  Repair  Support  Program.” 
1  October  1983  and  Army  Regulation  750-43,  "Test,  Measurement,  and  Diagnostic  Equipment,” 
1  April  1984.  Both  the  technical  direction  and  the  management  and  control  of  calibration  and  repair 
support  of  all  general-purpose  and  some  special-purpose  TMDE  is  concentrated  in  USATSG  (The 
decision  for  USATSG  to  support  selected  items  of  special-purpose  TMDE  is  made  by  the  materiel 
developer  of  the  supported  prime  equipment  and  USATSG  based  on  operational  constraints  and  cost 
effectiveness.)  USATSG  has  technical  control  of  the  TMDE  support  program  Army  wide,  it  exercises 
command  and  control  of  all  ACRCs  and  designated  ICRCs:  it  manages  and  operates  the  Primary 
Standards  Laboratory;  and  it  is  responsible  for  development  of  calibration  procedures  and  the  acquisi¬ 
tion  of  calibration  equipment.  It  also  publishes  Technical  Bulletin  4,3- 180,  "Calibration  Require¬ 
ments  for  the  Maintenance  of  Army  Materiel”  (current  issue  dated  15  April  1983),  which  identifies 


the  calibration  and  repair  requirements  for  TMDE  used  in  the  field.  (The  current  issue  lists  about 
4,200  line  items  with  calibration  required,  and  2,300  line  items  with  no  calibration  required.) 
USATSG  is  directed  by  the  Commanding  General,  AMC,  through  the  Executive  Director  for  TMDE, 
who  is  responsible  for  managing  the  acquisition,  logistics,  and  financial  management  of  all  Army 
TMDE.  The  major  commands  that  use  the  calibration  services  provided  by  USATSG  elements  are 
responsible  for  maintaining  a  TMDE  management  program,  providing  facilities  and  support  to 
calibration/repair  activities,  establishing  calibration/repair  support  priorities  for  subordinate  units, 
and  coordinating  their  war  plans  with  AMC  to  ensure  that  calibration  and  repair  support  is  ade¬ 
quately  addressed. 

In  summary,  the  Army  has  consolidated  the  calibration  and  repair  support  of  most 
TMDE  under  a  single  organization  that  is  responsible  for  technical  direction  as  well  as  management 
and  control. 

Navy  The  Navy’s  principal  policy  document  on  calibration  and  repair  of  test  equipment 
is  NAVELEX  Instruction  4355.2,  "Department  of  the  Navy  Metrology  and  Calibration  (METCAL) 
Program”  (draft  April  1984)  The  Metrology  Engineering  Center,  Pomona,  California,  under  the 
administrative  command  of  the  Naval  Weapons  Station,  Seal  Beach,  California,  a  NAVSEA  field 
activity,  is  responsible  for  technical  direction  of  the  Navy’s  METCAL  program  It  develops  and 
evaluates  calibration  standards  and  equipment,  maintains  the  associated  ILS  plans,  and  promulgates 
the  "Navy  Calibration  Equipment  List  ”  (NAVAIR  17-35  NCE-1,  NAVELEX  0969-LP -170-5010,  and 
NAVSEA  OD48939)  It  develops  or  approves  calibration  procedures  and  documentation  for  all  Navy 
calibration  activities,  and  maintains  the  list  of  authorized  procedures,  "Metrology  Requirements 
List”  (NAVAIR  17  35  MTL-1,  NAVELEX  0967-133-2010,  and  NAVSEA  OD45845).  It  also  deter 
mines  and  adjusts  calibration  intervals  for  test  equipment  and  standards  to  meet  required  reliability 
levels.  Overall  management  and  control  of  the  METCAL  program  is  exercised  by  NAVELEX  (TAMS 
Program  Office,  ELEX-08T).  It  has  approval  authority  over  the  establishment,  disestablishment,  or 
expansion  of  any  Type  I,  II,  or  III  calibration  laboratory.  (Type  IV  Fleet  Calibration  Activities  are 
established  and  approved  by  the  Systems  Commands.)  The  Systems  Commands  are  responsible  for 


calibration  and  calibration  support  of  all  test  equipment  under  their  cognizance.  NAVAIR’s 
responsibilities  include  all  aviation-associated  test  equipment;  NAVSEA  is  responsible  for  all 
shipboard  hull/mechanical/electric  and  combat  system  test  equipment;  and  NAVELEX  is  responsible 
for  shipboard  and  shore  electronic  test  equipment.  They  operate  assigned  laboratories  and  provide 
funding  for  research  and  development,  procurement,  and  logistic  support  (including  calibration  and 
repair)  of  metrology  standards  and  for  calibration  and  incidental  repair  of  Fleet  test  equipment 
beyond  the  capability  or  capacity  of  the  Fleet. 

In  contrast  to  the  Army,  the  Navy  has  separated  responsibility  for  the  calibration  and 
repair  (other  than  incidental)  of  test  equipment.  Similarly,  the  technical  direction,  management,  and 
operation  of  calibration  laboratories  are  separate  and  performed  by  numerous  organizations. 

Air  Force.  The  Air  Force’s  principal  policy  documents  are  Air  Force  Regulation  74-2, 
"Air  Force  Metrology  and  Calibration  (METCAL)  Program,”  8  February  1983  and  T  O  00-20-14,  "Air 
Force  Metrology  and  Calibration  Program,”  15  May  1982.  Technical  direction  as  well  as  manage¬ 
ment  of  the  Air  Force's  METCAL  program  is  concentrated  in  AGMC,  subject  to  the  overall  oversight 
and  policies  established  by  AFLC  (Deputy  Chief  of  Staff  for  Maintenance).  AGMC  operates  the  Air 
Force's  Type  I  laboratory  It  determines  which  bases  and  installations  are  authorized  a  PMEL,  it 
establishes  and  adjusts  calibration  intervals;  it  prepares  and  publishes  technical  orders  for  cali¬ 
brating  reference  standards,  precision  measurement  equipment  (PME),  and  ATE;  it  maintains  a 
system  for  evaluating  PMELs;  and  it  reviews  prime  equipment  measurement  requirements  during 
development  It  also  determines  requirements  and  initiates  procurement  action  for  new  standards 
and  calibration  equipment.  In  contrast,  AFSC  is  responsible  for  identifying  test  and  measurement 
requirements,  obtaining  Support  Equipment  Recommendation  Data,  and  developing  or  providing  any 
new  peculiar  PME.  The  ALC  system  manager  is  responsible  for  development  or  procurement  of  any 
common  PME,  if  required,  before  new  or  modified  systems  are  operational.  AFSC  is  also  responsible, 
in  coordination  with  AFLC  and  AGMC,  for  the  development  of  new  equipment  required  to  support 
new  PME  The  major  commands  are  responsible  for  maintaining  and  operating  PMELs  to  calibrate 
and  repair  all  common  and  designated  peculiar  PME  of  host,  tenant,  and  off-base  supported  Air  Force 
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activities  (including  those  of  Air  National  Guard  and  Air  Force  Reserve).  They  are  also  responsible 
for  assisting  supported  activities  with  calibration  and  repair  of  peculiar  PME  not  specified  as  a  PMEL 
responsibility. 


The  Air  Force  has  concentrated  the  technical  direction  and  management  of  calibration 
in  one  organization,  with  the  major  commands  (operating  forces)  responsible  for  operating  the  cali¬ 
bration  laboratories  that  also  provide  repair  support  for  most  test  equipment 

Observations.  The  extent  of  control  by  the  user  (operating  forces)  over  calibration  ser¬ 
vices  has  been  at  issue  for  several  years.  Currently,  the  user  has  no  control  in  the  Army,  some  control 
in  the  Navy,  and  considerable  control  in  the  Air  Force.  Furthermore,  the  Military  Services  also  differ 
substantially  with  respect  to  their  treatment  of  test  equipment  repair,  particularly  between  the  Navy 
and  both  the  Army  and  Air  Force.  The  biggest  complaint  in  the  Fleet  with  regard  to  test  equipment  is 
the  weak  repair  support  and  the  lengthy  delays  resulting  from  test  equipment  that  is  turned  in  for 
calibration  being  returned  by  the  laboratory  for  repair  elsewhere  before  it  is  accepted  for  calibration. 

Maintenance  Concept 

By  policy,  the  Army  uses  the  LSA  to  develop  a  cost-effective  maintenance  concept  for 
test  equipment,  including  commercial  off-the-shelf  ETE,  in  accordance  with  MIL-STD-1388  Army 
policy  also  requires  the  U  S.  Army  Communications  Command,  U  S.  Army  Intelligence  and  Security 
Command,  and  AMC  to  determine  maintenance  float  requirements  for  TMDE  and  to  coordinate  with 
USATSG  that  such  floats  are  procured,  supplied,  and  distributed  to  ACRCs  Other  major  commands, 
however,  are  not  so  directed.  As  a  result,  U  S  Army,  Europe,  for  example,  has  not  established  a 
maintenance  float  except  for  some  radiac  items 

In  contrast,  the  Air  Force  requires  the  ALCs  to  perform  depot-level  maintenance  to 
restore  PME  to  a  serviceable  condition  There  is  little  consideration  given  to  other  factors  that  might 
influence  that  decision,  such  as  LSA,  cost  effectiveness,  complexity,  and  the  use  of  commercial  versus 
militarized  equipment.  The  Air  Force,  like  the  Army,  does  not  have  any  maintenance  float  for  this 
type  of  equipment 
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The  Navy  does  not  have  a  specific  policy  on  test  equipment  maintenance  although 
NAVAIR  has  been  trying  to  develop  a  policy  on  the  acquisition  and  life  cycle  support  of  commercial 
test  equipment  particularly  for  ATE.  In  practice,  the  Navy  appears  to  have  pursued  a  concept  of 
maximum  organic  maintenance  of  test  equipment,  which  has  created  a  conflict  with  the  long-term 
supportability  of  commercial  test  equipment.  As  a  result  of  this  conflict,  the  Navy  has  undertaken 
two  studies  to  explore  how  commercial  test  equipment  should  be  supported.  One  advocates  maximum 
commercial  support,1  the  other  advocates  organic  support.2 

The  Marine  Corps  has  always  insisted  on  maximum  organic  maintenance  capability  for 
test  equipment  although  local  commanders  have,  by  policy,  the  discretion  to  use  either  the  Marine 
orps  supply  system  or  local  commercial  parts  support. 

It  appears  that  the  cost  effectiveness  of  organic  support  of  commercial  test  equipment  is 
being  overestimated.  The  alternative  of  commercial  support,  using  float  items  to  protect  test  equip¬ 
ment  availability,  may  in  many  cases  be  the  most  cost-effective  solution,  especially  in  those  instances 
in  which  the  item  density  is  low  and  the  item  technology  is  high  (i.e.,  the  item  is  subject  to  rapid 
obsolescence).  These  differences  in  maintenance  concepts  clearly  illustrate  the  need  for  a  DoD  policy 
on  test  equipment  support. 

Calibration 

The  calibration  interval,  i.e.,  the  time  period  between  successive  calibrations  of  the  same 
test  instrument,  is  defined  as. 

.  .  .  that  period  of  time,  based  on  an  8  hour  operating  day,  that  equipment 
has  not  degraded  from  the  limits  required  by  the  detailed  specification 
[MIL-T-28800C,p  1361 


•Dialectic  Systems  Corporation,  Commercial  Test  Equipment  Report  Phase  1  (Draft) 
(Washington,  D  C.:  Naval  Air  Systems  Command,  13  August  1982). 

2Harris  Corporation,  CTE  Repair  Study  Pha~  H  Report  (Washington.  D  C.  Naval  Air 
Systems  Command,  14  February  1985). 
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The  same  document  recommends  the  initial  calibration  interval  for  new  test  equipment  if  the  perfor¬ 
mance  specification  does  not  specify  one: 

When  the  detailed  specification  does  not  require  a  specific  calibration  inter¬ 
val,  the  calibration  interval  shall  be  nine  months,  based  on  an  equipment 
operating  time  of  1500  hours.  [MIL-T-28800C,  p.  80] 

The  Military  Services  use  a  common  approach  to  determining  and  adjusting  calibration 
intervals  of  test  equipment.  Initially,  for  new  equipment,  the  interval  may  be  set  in  accordance  with 
manufacturer  recommendations  or  like  equipment  in  the  inventory.  (The  purchase  descriptions  for 
commercial  off-the-shelf  ETE  normally  do  not  include  calibration  interval  requirements.)  Each  of  the 
Military  Service’s  METCAL  program  includes  a  comprehensive  system  for  monitoring  the  number 
and  percentage  of  test  instruments  that  are  found  out-of-tolerance  when  they  are  turned  in  for  cali¬ 
bration.  Based  on  such  field  experience,  calibration  intervals  are  adjusted  until  they  meet  individual 
Military  Service  criteria.  For  example,  the  Air  Force’s  criterion  is  an  end-of-period  within-tolerance 
rate  (also  called  reliability)  of  85  percent,  which  corresponds  to  an  average  rate  of  95  percent  over  the 
period  between  calibrations.  The  Navy  and  Army  require  an  average  rate  of  85  percent,  which  corre¬ 
sponds  to  an  end-of-period  within-tolerance  rate  of  approximately  72  percent  for  Navy  and  75  percent 
for  Army. 

Furthermore,  these  rules  have  traditionally  been  applied  to  each  make/model  of  test 
equipment,  with  little  effort  to  discriminate  among  the  different  makes/models  of  test  equipment  and 
give  each  make/model  its  own  reliability  target  as  a  function  of  criticality  (e  g.,  safety-of-flight 
measurements)  and  out-of-tolerance  rate.  There  are  indications  now  that  the  METCAL  community  is 
beginning  to  recognize  that  this  approach  may  not  be  appropriate.3 

The  issue  of  differential  reliability  targets  seems  more  important  than  the  issue  of  cali¬ 
bration  interval  differences,  which  has  been  the  object  of  many  studies  over  the  past  several  years 

3Steven  R.  Dwyer,  "Reliability  Targets  for  a  Calibration  Program,”  1984  Proceedings  Annual 
Reliability  and  Maintainability  Symposium  (New  York:  The  Institute  for  Electrical  and  Electronic 
Engineers,  Inc  ,  1984) 


Ideally,  the  methodology  for  determining  differential  reliability  targets  to  minimize  calibration  costs 
and  maximize  reliability  should  be  standard  throughout  the  DoD;  however,  the  resulting  calibration 
intervals  for  like  equipment  may,  of  course,  be  different  among  the  Military  Services. 

ETE  CALIBRATION  AND  REPAIR 

This  section  describes  the  test  equipment  calibration  and  repair  organizations  and  capabilities 
within  the  Military  Services. 


The  Army’s  organizational  structure  for  TMDE  calibration  and  repair  support  is  illus¬ 
trated  in  Figure  3-2.  Support  in  CONUS  is  divided  between  two  organizations  subordinate  to 
USATSG.  the  U.S.  Army  TMDE  Support  Activity  CONUS  (USATSAC),  which  is  a  civilian  TDA 
organization  that  provides  support  to  TRADOC,  the  AMC  industrial  base,  reserve  components,  and 


FIGURE  3-2.  ARMY  TMDE  SUPPORT 


nondivisional  elements  of  FORSCOM;  and  the  95th  Maintenance  Company  (TMDE),  which  is  a 
military-staffed  organization  with  headquarters  at  Redstone  Arsenal,  Alabama,  providing  support  to 
the  10  FORSCOM  divisions  located  at  9  CONUS  installations. 

The  USATSAC  comprises  Five  (six  in  the  near  future)  ICRCs  that  are  tailored  to  the  spe¬ 
cific  support  requirements  of  the  installations  at  which  they  are  located  and  six  ACRCs  that  are 
located  at  Army  depots  (Anniston,  Alabama,  Lexington-Bluegrass,  Kentucky;  Letterkenny  at 
Chambersburg,  Pennsylvania;  Pueblo,  Colorado;  Sacramento,  California;  and  Tobyhanna, 
Pennsylvania).  Each  of  the  ICRCs  has  an  ICL,  and,  altogether,  they  operate  34  separate  internal 
TMDE  support  activities.  Each  of  the  ACRCs  has  an  ACL  and  is  responsible  for  customer  support  in  a 
specific  geographic  region  (Figure  3-3).  That  support  is  provided  by  61  ATSTs,  which  operate  either 


FIGURE  3-3.  U  S.  ARMY  CALIBRATION  ACTIVITIES  NONDEPLOYABLE 


SOURCE  Michael  C.  Sandusky.  Final  Comprehensive  Report  on  the  U  S.  Army  Calibration 
Program.  Vol  1  (Alexandria,  Virginia:  Headquarters  AMC,  Office  of  the  Deputy  Executive  Director 
for  TMDE,  July  1984). 
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at  fixed  sites  or  as  mobile  teams  for  on-site  support.  These  ATSTs  do  not  have  a  wartime  deployment 
role. 


The  95th  Maintenance  Company,  in  contrast,  is  a  deployable  TMDE  support  organi¬ 
zation.  It  comprises  an  ACL  and  17  ATSTs,  with  1  or  more  attached  to  each  CONUS  division  and  the 
1st Corps  Support  Command  at  Fort  Bragg,  North  Carolina  (Figure  3-4).  In  wartime,  one  ATST 
would  deploy  with  each  FORSCOM  and  National  Guard  division. 


FIGURE  3-4.  95thMAINTENANCE  COMPANY  (TMDE)  ATST  LOCATIONS 


(21»t  Support  Command).  Each  company  has  one  ACL  located  at  company  headquarters  and  several 
ATSTs  that  operate  either  at  fixed  sites  or  as  mobile  teams  visiting  supported  units  at  120-day 
intervals  (Calibration  intervals  of  Army  TMDE  are  in  multiples  of  120  days  currently,  with  studies 
in  progress  to  extend  these  intervals  to  180-day  multiples.)  A  total  of  45  ATSTs  operate  at  or  from  the 
21  sites  shown  in  Figure  3-5.  The  area  support  concept  used  in  peacetime  would  revert  to  a 
unit-alignment  concept  in  wartime,  with  fully  mobile  ATSTs  attached  to  division  support  commands, 
Corps  Support  Commands,  and  Theater  Army  Area  Commands. 

FIGURE  3-5.  517th  MAINTENANCE  BATTALION  (TMDE).  EUROPE 


•  HO  S17  MAINT  BN  (TMOCI 
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SOURCE:  Michael  C  Sandusky.  Final.  Comprehensive  Report  on  the  U  S.  Army  Calibration 
Program.  Vol.  I  (Alexandria,  Virginia:  Headquarters  AMC,  Office  of  the  Deputv  Executive  Director 
for  TMDE,  July  1984) 


The  primary  mission  of  the  74th  Maintenance  Battalion  (TMDE)  is  to  support  the 
2nd  Infantry  Division  in  Korea  It  has  only  one  maintenance  company  with  one  ACL,  located  at  Camp 
Carroll,  Korea,  and  14  ATSTs  located  in  Korea,  Japan,  and  Okinawa  (see  Figure  3-6)  The  battalion 


also  has  three  ATSTs  to  provide  support  to  the  Western  Command  and  the  25th  Infantry  Division  in 
Hawaii. 


FIGURE  3-6.  74th  MAINTENANCE  BATTALION  (TMDE).  PACIFIC 


SOURCE:  Michael  C  Sandusky.  Final  Comprehensive  Report  on  the  U  S.  Army  Calibration 
Program.  Vol.  1  (Alexandria,  Virginia.  Headquarters  AMC,  Office  of  the  Deputy  Executive  Director 
for  TMDE,  July  1984) 


The  Army  has  140  ATSTs,  61  of  which  are  staffed  with  civilian  and  79  with  military 
personnel.  Each  team  is  equipped  with  a  secondary  transfer  standards  set  in  an  expansion  van, 
mounted  on  a  5-ton,  10-wheel  truck.  The  set  comes  in  two  configurations  — AN/GSM-286  (consisting 
of  direct  current/low-frequency  and  physical  standards)  and  AN/GSM-287  (same  as  AN/GSM-286 
supplemented  with  additional  standards,  including  microwave,  in  a  second  truck-mounted  van). 
Each  team  is  mobile,  with  power  provided  by  a  trailer-mounted  generator  The  AN/GSM-286  team 
has  five  billets,  four  of  which  are  calibration  specialists.  The  AN/GSM-287  team  is  authorized 


seven  billets,  including  six  calibration  specialists;  in  addition,  a  warrant  officer  is  authorized  for 
every  three  teams  to  serve  as  a  supervisor. 


The  overseas  ATSTs  and  ACLs  are  augmented  by  civilians  (TDA  augments  or  con¬ 
tractors).  The  extent  of  augmentation  is  shown  in  Table  3-2.  The  organizations  above  the  horizontal 
line  in  the  table  do  not  have  a  deployment  mission,  while  those  below  have  such  a  mission.  The  term 
"technician”  refers  to  those  military  and  civilian  personnel  doing  hands-on  work,  excluding 
administrative,  supply  support,  and  other  overhead  personnel.  The  entry  for  the  Army  National 
Guard  refers  to  the  TMDE  support  activities  located  at  Combined  State  Maintenance  Shops  (CSMSs). 
(About  20  percent  of  TMDE  calibration/repair  workload  is  beyond  the  National  Guard’s  capability 
and  is  performed  by  USATSAC,  except  in  Puerto  Rico  where  the  excess  workload  is  performed  by  the 
Navy  under  an  ISSA.)  As  shown  in  the  table,  the  overseas  TMDE  support  units  make  extensive  use  of 
civilians  to  perform  their  workload.  Moreover,  most  are  contractors  or  local  nationals.  (The  latter 
category  includes  direct  hires,  indirect  hires  through  host  nation  support  agreements,  and  Civilian 
Support  Group  personnel  that  are  quasi-military  personnel.) 

The  actual  use  of  "nonorganic”  personnel  to  calibrate  and  repair  TMDE  is  even  more 
prevalent  than  shown  in  Table  3-2.  A  recent  Army  audit  revealed  that  approximately  two-thirds  of 
the  TMDE  repair  workload  is  being  performed  by  the  nonorganic  personnel,  who  comprise  only 
17  percent  of  total  staffing.* 

More  recent  data  (May  1985),  show  that  the  517th  Maintenance  Battalion  is  now  sup¬ 
ported  by  about  130  "nonorganic”  personnel  as  follows: 

•  Department  of  the  Army  civilians:  9 

•  Civilian  Support  Group:  26 

•  Local  nationals:  54 


4U  S  Army  Audit  Agency,  Report  of  Audit.  Test.  Measurement,  and  Diagnostic  Equipment 
Support.  L'.S  Army  Europe  and  Seventh  Army.  Audit  Report  EU85-201  (Alexandria,  Virginia: 
Department  of  the  Army  Office  of  the  Auditor  General,  10  October  1984) 
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TABLE  3-2.  USE  OF  CIVILIANS  IN  U  S  ARMY  TMDE  CALIBRATION/REPAIR  SUPPORT 


TYPE  OF  CALIBRATION  ACTIVITY 

CIVILIAN 

TECHNICIANS 

MILITARY 

TECHNICIANS 

Authorized 

On-Hand 

Authorized 

On-Hand 

Primary  Standards  Laboratory 

36 

33 

4 

4 

ACRCs  under  USATSAC 

279 

265 

45 

79 

ICRCs  under  USATSAC 

63 

56 

46 

55 

AGMC  Internal  Facilities 

363 

321“ 

38 

34 

Army  National  Guard  (CSMS) 

148 

130 

0 

0 

95th  Maintenance  Co.,  Redstone  Arsenalb 

0 

0 

123 

199 

517th  Maintenance  Bn  (TMDE),  Germany13 

62 

59“ 

265 

297 

74th  Maintenance  Bn  (TMDE),  Koreab 

36 

36d 

82 

101 

TOTAL 

987 

900“ 

603 

769 

“Includes  50  contractor  personnel. 

bUnits  with  wartime  deployment  mission  for  TMDE  support. 

Clncludes  27  contractor  personnel,  14  local  nationals,  and  18  Civilian  Support  Group  per¬ 
sonnel. 

^Includes  14  local  nationals  and  12  contractor  personnel;  only  the  TMDE  support  detachment 
in  Hawaii  uses  Department  of  the  Army  civilians. 

“Includes  89  contractor,  28  local  national,  and  18  Civilian  Support  Group  personnel 

SOURCE:  Michael  C.  Sandusky,  Final  Comprehensive  Report  on  the  U  S.  Army  Calibration 
Program.  Vol  1  (Alexandria,  Virginia:  Headquarters  AMC,  Office  of  the  Deputy  Executive 
Director  for  TMDE,  July  1984). 

•  Federal  Electric,  Inc.:  31  (25  supporting  the  517th  and  6  performing  TMDE  support 
for  the  Area  Maintenance  Support  Facility  at  Mannheim) 

•  Three  other  contractors  supporting  the  test  equipment  for  specific  systems 

•  NATO  [North  Atlantic  Treaty  Organization)  Maintenance  and  Supply  Activitv 
(NAMSA):  6 

The  recent  reorganizations  of  TMDE  support  in  Europe,  including  the  consolidation  of 
calibration  and  repair  under  the  U.S.  Army  TMDE  Support  Activity  Europe  in  October  1979  and  the 
conversion  of  that  TDA  unit  into  the  517th  Maintenance  Battalion  on  1  February  1983,  have  reduced 
backlogs  from  a  peak  of  8,000  items  in  June  1981  to  around  2,000  in  1984  and  shortened  repair 
turnaround  times  from  60  days  in  April  1981  to  the  mid-20s  in  1984.  Both  of  these  improvements, 
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however,  can  be  traced  to  the  performance  of  nonorganic  personnel.  According  to  personnel  from  the 
517lh,  elimination  of  those  nonorganic  personnel  would  necessitate  an  increase  of  118  enlisted  person¬ 
nel.  An  additional  40  technicians  would  be  required  if  contractor  support  for  the  fielding  of  new 
equipment  were  also  eliminated. 

A  recent  study  conducted  by  USATSG,  using  a  simulation  model,  concluded  that  the 
Army’s  TMDE  support  requirements  in  wartime  could  be  accommodated  by  142  ATSTs  (75  AN'/ 
GSM-286  and  67  AN/GSM-287)  rather  than  the  215  previously  estimated.5  The  reduction  resulted 
from  a  more  precise  estimate  of  wartime  workload,  consolidating  nondivisional  microwave 
calibration  support,  and  introducing  automation  of  meter  calibration.  (Multimeters  and  voltage 
meters  account  for  28  percent  of  the  Army’s  TMDE  inventory.)  The  Army  is  now  planning  a  phased 
modernization  of  its  present  inventory  of  143  AN/GSM-286  and  AN/GSM-287  sets  to  replace  obsolete 
equipment,  add  instrument  controllers,  and  include  automated  meter  calibration. 

Navy 

ETE  support  in  the  Navy  is  divided  between  owning  organizations,  which  have  the 
primary  repair  responsibility,  and  calibration  laboratories  or  activities,  which  calibrate  the  ETE  If 
the  repairs  are  beyond  the  capability  of  the  owning  organization,  then  that  organization  requests 
assistance  from  activities  that  can  perform  the  repairs,  such  as  tenders,  Naval  Shipyards,  Weapons 
Stations,  or  NARFs. 

The  Navy  organization  for  calibrating  ETE  is  much  more  formal  than  its  organization 
for  repairing  it  (see  Figure  3-7).  Type  Commanders,  such  as  the  Commander,  Naval  Air  Forces 
Atlantic,  or  Commander,  Naval  Surface  Forces  Pacific,  are  assigned  responsibility  for  Fleet 
calibration  laboratories  These  laboratories  are  located  aboard  tenders,  repair  ships,  and  other  ships 
with  intermediate-maintenance  capabilities  (such  as  carriers)  and  are  also  located  ashore  at  SIMAs 
and  Aircraft  Intermediate  Maintenance  Departments.  Calibrations  performed  by  these  laboratories 

5Michael  C  Sandusky,  Final  Comprehensive  Report  on  the  L'.S.  Army  Calibration  Program. 
Vol  1  (Alexandria,  Virginia:  Headquarters  AMC,  Office  of  the  Deputy  Executive  Director  for 
TMDE,  July  1984) 
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FIGURE  3-7.  NAVY  ETE  SUPPORT 


are  managed  using  MEASURE.  Under  this  system,  equipment  is  called  in  for  calibration  automat¬ 
ically  at  scheduled  calibration  intervals.  (The  Army  and  Air  Force  have  comparable  recall  systems 
for  managing  and  monitoring  calibration.)  Type  Commanders  are  also  responsible  for  Type  IV  Fleet 
Calibration  Activities  that  are  operated  by  many  units  for  organizational  support  of  their  own  test 
equipment.  Both  types  of  laboratories  perform  repairs  incidental  to  the  calibration,  usually  those 
that  can  be  accomplished  within  1  hour  and  require  no  material.  The  Marine  Corps  calibrates  its  own 
equipment  using  either  calibration  laboratories  installed  in  mobile  calibration  complexes  or  field 
calibration  activities. 

Quantitative  data  on  the  support  of  test  equipment  were  not  available  from  the  Navy. 
However,  several  interviews  were  conducted  for  purposes  of  obtaining  an  overview  of  the  effective¬ 
ness  of  test  equipment  calibration  and  repair  support  in  the  Navy.  Among  those  interviewed  were 
Type  Commanders,  representatives  from  two  destroyer  squadrons,  and  supervisory  personnel  from 
tenders  and  SIMAs. 

The  Type  Commanders  believe  that  test  equipment  management  in  the  Navy  needs  to 
be  improved.  They  are  missing  some  test  equipment;  the  repair  pipelines  are  excessive;  and  some  of 
the  ATE  is  antiquated  and  unsupportable  (one  example  mentioned  was  the  AN/AWM-23,  which 
supports  the  F-14  radar)  They  also  stressed  the  increasing  need  for  repair  support  because  of  a 
dramatic  increase  in  damaged  or  abused  general-purpose  test  equipment  received  at  calibration 
facilities.  (This  problem,  attributed  to  inadequately  trained  personnel,  was  first  called  to  the 
attention  of  the  Chief  of  Naval  Technical  Training  by  the  Commander  in  Chief,  Atlantic  Fleet  in  late 
1981 .6) 

The  destroyer  squadron  staff  observed  that  management  of  ETE  is  "out  of  hand.”  On  the 
East  Coast,  calibration  requests  are  normally  submitted  via  the  Readiness  Support  Group  (Norfolk) 
to  a  tender  or  SIMA,  but  those  requests  are  rejected  when  the  ETE  is  not  operable.  There  is  no 
replacement  system  for  ETE  that  cannot  be  economically  repaired.  Furthermore,  repair  activities 

•’Commander  in  Chief,  Atlantic  Fleet  message  to  Chief  of  Naval  Technical  Training,  Subject: 

"Training  of  Technical  Ratings  for  Fleet  Maintenance,”  date  time  group  071410Z,  October  1981 
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may  dispose  of  ETE  if  they  cannot  economically  repair  it,  but  they  do  not  notify  the  ship  that  they 
have  taken  such  an  action.  When  the  ship  finds  that  the  equipment  has  been  disposed,  it  has  to 
procure  replacement  ETE  using  its  own  limited  funds.  The  destroyer  squadron  staff  also  noted  that 
deploying  ships  never  have  all  of  their  test  equipment  allowance  aboard,  even  after  obtaining  as 
much  as  possible  from  other  ships.  They  are  invariably  10  to  30  items  short,  with  most  of  those  being 
low-density  items  (those  for  which  the  ship  is  authorized  only  1  or  2).  The  missing  items  affect  the 
crew’s  ability  to  perform  planned  maintenance  while  underway  because  the  maintenance  require¬ 
ments  cards  specify  particular  ETE.  Although  it  is  frequently  possible  to  perform  the  maintenance 
using  substitute  ETE,  that  approach  requires  special  skills  that  may  not  be  available.  Finally,  the 
destroyer  squadron  staff  also  noted  that  approximately  70  percent  of  its  GPETE  is  obsolete  and  needs 
to  be  replaced 

The  tender  personnel  stated  that  the  turnaround  time  for  GPETE  repair  typically 
averages  30  days,  while  that  for  special-purpose  ETE,  which  goes  to  a  shore-based  repair  facility  or 
depot,  frequently  is  as  long  as  l  year  They  also  noted  that  the  surface  Navy  does  not  have  a  technical 
training  course,  as  the  air  community  does,  for  the  training  of  test  equipment  repair.  Any  "A”  School 
graduate  in  one  of  the  technical  ratings  who  completes  the  tri-Service  calibration  course  at  Lowry 
Air  Force  Base,  Colorado  becomes  a  calibration/repair  technician  without  further  training  (In  the 
Navy,  NARFs  provide  a  2-week  course  for  ETE  repair  ) 

Air  Force 

The  Air  Force’s  test  equipment  support  management  structure  is  straightforward,  as 
shown  in  Figure  3-8  The  Metrology  Division  of  AGMC  is  the  central  Air  Force  authority  for 
calibration  procedures,  schedules,  and  equipment.  It  operates  the  Air  Force  Measurements 
Standards  Laboratory,  the  Air  Force’s  Type  I  laboratory,  and  it  exercises  technical  direction  and 
evaluation  authority  over  all  Air  Force  PMELs.  The  major  Air  Force  commands  are  responsible  for 
maintaining  and  operating  PMELs  at  selected  bases  and  installations  to  calibrate,  certify,  and  repair 
all  common  and  designated  peculiar  test  equipment  of  the  host,  tenant,  and  off-base  supported  Air 
Force  activities,  including  the  Air  National  Guard  and  Air  Force  Reserve  They  are  also  responsible 


for  operating  programs  for  PME  scheduling  and  maintenance  data  collection  and  for  assisting 
supported  activities  with  the  calibration  and  repair  of  peculiar  PME  (not  specified  as  a  PMEL 
responsibility). 


FIGURE  3-8.  AIR  FORCE  PME  SUPPORT 


In  addition  to  the  PMELs  supporting  the  operating  forces,  AFLC  operates  PMELs  that 
are  dedicated  to  the  support  of  its  depot  activities  (one  PMEL  per  ALC)  and  AFSC  operates  the  special 
PMELs  for  in-house  dedicated  support  to  its  research,  development,  test,  and  evaluation  activities. 
The  responsibilities  of  AFSC  and  AFLC  during  the  acquisition  process  to  ensure  that  test  equipment, 
calibration  procedures,  and  calibration  equipment  will  be  available  are  described  earlier  in  this 
report. 

The  134  Air  Force  PMELs  are  authorized  3. 100  personnel;  if  the  major  command 
functional  area  managers,  the  Air  Training  Command  PMEL  school  at  Lowry  Air  Force  Base,  the 
logistics  personnel  at  AFSC  and  AFLC.  and  AGMC  personnel  are  included,  then  the  total  METCAL 
community  numbers  slightly  over  4,000  people.  Volume  II  of  this  report  describes  the  Air  Force’s 


1981  improvement  program  that  focused  on  modernization  of  PMEL  equipment,  improvement  of 
management  procedures,  and  initiatives  in  the  personnel  area. 


Interservice  Support 

The  organizational  structure  of  the  Joint  Technical  Coordination  Group  for  Metrology 
and  Calibration  (JTCG-METCAL)  is  shown  in  Figure  3-9.  Since  1976,  the  JTCG-METCAL  has 
sponsored  a  number  of  consolidation  studies.  One  study  addressed  the  feasibility  of  consolidating  the 
Type  I  laboratories,  others  focused  on  the  economics  of  consolidating  calibration  activities  in  regional 
areas  without  degrading  mission  capabilities.  Such  studies  have  been  performed  for  the  Tidewater/ 
Norfolk  area,  Philadelphia  area,  Washington,  D.C.  area,  Northwest  area,  California,  and  Europe 
Few  of  the  recommendations  for  consolidating  calibration  activities  have  been  implemented,  how¬ 
ever.  The  study  of  European  activities  illustrates  some  of  the  difficulties  of  implementing  those  types 
of  recommendations. 

That  study,  "Consolidation  of  DoD  Calibration  Activities  in  Europe,”  was  conducted  in 
response  to  a  U  S.  Commander  in  Chief,  Europe  request  in  October  1975  that  the  Army  convene  and 
chair  an  ad  hoc  study  group,  with  Air  Force  and  Navy  representation,  to  determine  the  feasibility  and 
advisability  of  establishing  consolidated  calibration  facilities  within  the  European  Theater.  The 
ad  hoc  group  was  established  in  November  1975  and  requested  support  from  JTCG-METCAL.  In 
January  1976,  the  Consolidation  of  Calibration  Services  Subgroup  met  with  the  ad  hoc  group  and 
agreed  to  perform  the  study  that  was  subsequently  completed  in  September  1976.  The  subgroup 
recommended  a  concept  of  two  consolidated  calibration  laboratories  in  the  Federal  Republic  of 
Germany  (FRG): 

•  Consolidation  of  Air  Force  PMELs  at  Hahn  and  Bitburg  Air  Bases  into  one  expanded 
PMEL  at  Ramstein  Air  Base. 

•  Consolidation  of  Army  ACRCs  at  Pirmasens  and  Augsburg  into  one  expanded  ACRC 
at  Schwanheim.  (The  report  noted  that  Schwanbeim  was  not  the  optimal  location 
and  recommended  that  another  site  for  this  single  Army  laboratory  be  selected.) 

•  Implementation  of  ISSAs  between  the  Air  Force  and  the  Army,  with  the  Air  Force 
assuming  the  Army’s  calibration  workload  in  the  Kaiserslautern,  FRG  area,  Italy, 
and  Turkey  and  the  Army  assuming  the  Air  Force's  calibration  workload  in  the 
Frankfurt-Wiirzburg,  FRG  area,  using  its  ATSTs. 


FIGURE  3-9.  ORGANIZATION  CHART  JTCG-METCAL 


SOURCE:  Fred  B.  Seeley  (DARCOM  Member),  Joseph  T.  Siedlecki  (NMC  Member  and  Chair¬ 
man),  Seldon  W  McKnight  (AFLC  Member),  and  Major  M.  J  Murtaugh  (AFSC  Member),  "Study 
Plan  Joint  Technical  Coordinating  Group  on  Metrology  and  Calibration  (JTCG-METCAL),” 
Unpublished  Working  Paper,  22  November  1982. 


The  present  value  of  savings  (equipment/facility  costs  plus  annual  personnel,  transpor¬ 
tation,  and  support  costs)  on  a  5-year  discounted  basis  was  estimated  at  $3  million.  Importantly,  the 
subgroup  examined  the  potential  of  NAMSA  providing  calibration  support,  but  determined  that  such 
support  would  not  be  economic.  The  cost  comparison,  however,  was  made  between  a  fully  loaded 
hourly  rate  of  NAMSA  (including  burden,  general/administrative,  and  overhead  expenses)  and  direct 


charges  of  DoD  Components  (excluding  those  indirect  expenses).  As  indicated,  the  recommended  con¬ 
solidations  were  never  implemented. 

Observations 

In  comparing  the  different  management  approaches  to  test  equipment  support,  one 
striking  difference  is  the  extent  of  control  exercised  by  the  user,  i.e.,  commanders  of  the  operating 
forces.  While  calibration  of  test  equipment  is  offered  as  a  "free  service”  to  the  user  throughout  the 
DoD,  the  user  has  no  control  in  the  Army,  some  in  the  Navy,  and  a  significant  amount  in  the  Air 
Force. 

The  Army  has  made  great  progress  in  recent  years  toward  improving  its  calibration  and 
repair  support  of  common  ETE.  Although  there  may  yet  be  some  unresolved  problems,  the  Army  has 
overcome  most  of  its  test  equipment  support  shortfalls.  The  Air  Force  has  also  made  progress  by 
firmly  addressing  the  PMEL  problems  that  resulted  primarily  from  neglect.  In  comparison,  the  Navy 
has  not  significantly  improved  test  equipment  support  in  the  Fleet. 

ATE  CALIBRATION  AND  REPAIR 

The  nature  and  extent  of  the  problems  associated  with  ATE  support  have  been  well  docu¬ 
mented  in  numerous  studies.  The  Joint  Logistics  Commanders  (JLC)  Panel  on  Automatic  Testing 
sponsored  an  effort  by  the  NBS  to  "determine  and  develop  concepts  (policies,  standards,  techniques, 
practices,  and  procedures)  for  on-line  and  off-line  calibration  to  verify  performance  of  the  DoD  family 
of  ATE,  with  special  emphasis  to  be  placed  on  performance  verification  of  third-generation  and  later 
generations  of  ATE.”  However,  after  1  year,  that  effort  was  terminated  because  of  lack  of  funding.7 

The  NBS  study  included  a  limited  experimental  investigation  of  the  measurement  perfor¬ 
mance  of  two  ATE  stations.  The  results  showed  erratic  measurements,  distorted  signals,  large 
measurement  errors  (departures  from  nominal  values),  and  other  shortcomings.  At  the  same  time, 
the  results  demonstrated  both  the  need  for,  and  the  viability  of,  on-site  dynamic  testing  of  ATE 

7Barry  A.  Bell,  et  al.,  Automatic  Test  Equipment  Calibration/Performance  Verification  Eval¬ 
uation  and  Research  Program.  NBSIR82-2601,  Parts  I  and  II  (Washington,  D  C.:  U  S  Department 
of  Commerce,  December  1982). 
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system  performance  characteristics  at  the  stimulus/measurement  interface  connector,  using  cali¬ 
brated  "dynamic  transport  standards”  (DTS). 

The  NBS  recommended  development  of  an  ATE  calibration  approach.  That  approach  is  illus¬ 
trated  in  Figure  3-10.  ATE  systems  would  be  tested  and  calibrated  on  site  with  a  commercial  DTS 
configured  as  a  calibration  console  for  a  particular  class  of  ATE  (Item  5)  The  commercial  DTS  would 
periodically  be  calibrated  at  a  (military)  calibration  or  standards  laboratory  using  the  NBS  DTS 
(Item  2),  the  laboratory’s  working  standards  (Item  3),  and  calibrators  (Item  4)  The  NBS  DTS,  in 
turn,  would  be  periodically  calibrated  at  the  NBS. 


FIGURE  3-10.  NBS  PROPOSED  CONCEPT  OF  DYNAMIC  TRANSPORT  STANDARDS  (DTS) 
TO  SUPPLEMENT  PRESENT  ATE  CALIBRATION  SYSTEMS 


SOURCE:  National  Bureau  of  Standards,  Automatic  Test  Equipment  Calibration/Perfor¬ 

mance  Verification  Evaluation  and  Research  Program.  NBSIR  82-2601,  Part  II  (Washington,  D  C  : 
US.  Department  of  Commerce,  December  1982) 


In  its  report,  the  NBS  outlined  a  long-term  research  and  development  program  to  deveiop  a 
capability  for  supporting  present  and  future  ATE  systems  used  by  DoD,  noting  that  the  "current 


system  is  not  adequate  to  develop  standards  and  traceability  requirements  for  state-of-the-art  ATE 
used  by  DoD.”  JLC  support  of  this  program  was  terminated  in  1982. 

PEACETIME  VERSUS  WARTIME  SUPPORT 

The  adequacy  of  present  test  equipment  support  capabilities  is  difficult  to  ascertain  because 
the  Military  Services,  with  one  exception,  do  not  collect  or  monitor  data  on  the  availability  of  operable 
test  equipment  at  the  using  maintenance  units.  (The  exception  is  San  Antonio  ALC’s  ATE  manage¬ 
ment  information  system.)  The  Joint  Chiefs  of  Staff  Unit  Status  Reporting  System  is  primarily 
designed  for  combat  units  rather  than  maintenance  units,  so  test  equipment  is  not  considered  a 
reportable,  pacing  item.  The  data  bases  established  by  the  METCAL  organizations  are  designed  to 
monitor  calibration  intervals,  not  test  equipment  availability.  Similarly,  the  developing  manage¬ 
ment  information  systems  for  test  equipment  are  designed  primarily  to  track  inventory  status. 

One  of  the  primary  performance  indicators  for  calibration  and  repair  shops  is  turnaround  time 
(TAT).  The  available  data  suggests  that  substantial  progress  is  being  made  to  reduce  TAT,  particu¬ 
larly  by  the  Army  in  Europe  where  the  average  repair  TAT  of  60  days  in  March  1981  reduced  to 
26  days  by  September  1984.  Similarly,  the  average  calibration  TAT  was  9  days  in  March  1981  and 
was  reduced  to  5  days  in  September  1984. 

Although  some  of  the  major  shortfalls  have  been  addressed,  two  ETE  support  problems  remain: 
(1)  shortages  of  experienced  technicians  (eg.,  over  60  percent  of  the  PMEL  work  force  in  the  Air  Force 
has  less  than  4  years  of  experience,  the  Army’s  517th  Maintenance  Battalion  claims  it  is  158  enlisted 
personnel  short  of  those  necessary  to  perform  its  mission  under  current  operating  conditions  without 
contractor  support)  and  (2)  shortages  of  spare  parts.  The  latter  is  one  of  the  primary  reasons  for 
excessive  TAT  for  test  equipment  repair. 

Overall,  test  equipment  support  may  be  considered  adequate  to  meet  peacetime  readiness 
requirements,  but  its  capability  to  meet  wartime  sustainability  requirements  is  questionable,  pri¬ 
marily  because  the  Military  Services  have  not  published  policies  or  plans  for  wartime  test  equipment 
support.  Some  of  the  Army’s  ACRC  supervisors  speculate  that  in  wartime  all  scheduled  calibrations 
would  be  stopped  and  that,  after  a  resulting  short  dip  in  the  workload,  test  equipment  repair 


workload  would  sharply  increase  from  peacetime  levels.  They  believe  that  the  result  would  be 
excessive  TATs  because  of  shortages  of  spare  parts  and  the  inability  to  compensate  because  the 
supported  maintenance  units  do  not  have  float  equipment.  In  contrast,  some  Air  Force  PMEL  techni¬ 
cians  assert  that  scheduled  calibrations  of  test  equipment  will  become  even  more  important  to  ensure 
peak  performance  of  supported  weapons  systems,  with  little  change  in  the  mix  (calibration  versus 
repair)  but  a  steep  increase  in  the  total  workload.  The  Navy  and  Marine  Corps  calibration  activities 
did  not  speculate  on  their  wartime  workload. 

Looking  at  the  issue  of  wartime  test  equipment  support  strictly  from  a  metrology  point  of  view, 
the  much  more  intense  use  of  test  equipment  would  mandate  decreasing  the  calibration  intervals  to 
maintain  the  same  within-tolerance  rate  as  in  peacetime.  As  a  result,  calibration  workload,  provided 
scheduled  calibrations  continue,  would  increase  sharply,  especially  in  the  Army  Separate  cali¬ 
bration  intervals  for  wartime,  however,  have  not  been  established  by  any  of  the  Military  Services. 
More  importantly,  if  the  peacetime  calibration  intervals  are  adhered  to  under  wartime  operating 
tempos,  the  within-tolerance  rate  of  test  equipment  would  plummet  to  a  level  at  which  the  test  equip¬ 
ment  could  not  be  used  in  weapons  system  maintenance. 

MANAGEMENT  AND  SUPPORT  OF  CALIBRATION  EQUIPMENT 

As  noted  in  Chapter  1,  this  study  does  not  extend  to  the  management  and  support  of  calibration 
equipment  but  rather  is  limited  to  test  equipment  used  in  weapons  system  maintenance.  Never¬ 
theless,  calibration  equipment  is  merely  another  category  of  test  equipment,  and  a  number  of  issues 
related  to  calibration  equipment  warrant  additional  attention:  (l)the  potential  for  much  more 
consolidation  than  has  been  achieved  over  the  past  years;  (2)  the  monitoring  of  the  materiel  condition 
of  calibration  standards;  (3)  the  modernization,  including  automation,  of  calibration  equipment; 
(4)  Joint  Service  standardization  of  calibration  procedures,  calibration  equipment,  and  calibration 
intervals,  including  those  for  modern  ATE;  and  (5)  the  lag  in  research  and  development  funding  for 
metrology  and  calibration.  Many  of  these  issues  appear  to  warrant  attention  by  the  JTCG-METC  AL. 


4.  ACQUISITION  AND  SUPPORT  OF  TEST  PROGRAM  SETS 


This  chapter  describes  the  TPS  acquisition  process  and  the  procedures  used  in  the  development 
and  life  cycle  support  of  test  programs.  Its  purpose  is  to  discuss  some  of  the  factors  that  may  explain 
the  problems  in  developing  and  fielding  TPSs  and  to  point  to  areas  in  need  of  increased  emphasis. 
BACKGROUND 

TPSs  represent  a  unique  category  of  test  equipment  because  they  consist  primarily  of  software, 
not  hardware.  Although  TPSs  are  essential  to  the  weapons  system  support  mission  of  the  ATE,  there 
is  no  consensus  on  whether  the  DoD  directive  pertaining  to  the  management  of  computer  software, 
other  than  general-purpose  commercial  automatic  data  processing,  resources  (DoD  Directive  5000  29, 
"Management  of  Computer  Resources  in  Major  Defense  Systems,”  26  April  1976)  does  or  should  apply 
to  TPSs. 

TPS  is  defined  in  MIL-STD-1309C  as  follows: 

The  combination  of  operational  test  program,  operational  test  program 
instruction,  and  interconnection  device  or  signal  conditioning  circuitry  that 
together  allow  an  ATE/TMDE  to  perform  the  test  necessary  to  check  and 
diagnose  a  UUT  [unit  under  test]. 

Test  programs  are  used  both  in  the  manufacturing  process  and  in  field  service  to  enable  auto¬ 
matic  testing  for  quality  control  (manufacturing)  and  for  fault  diagnosis  of  replaceable  assemblies 
and  modules  (maintenance).  The  traditional  test  philosophy  for  automatic  testing  in  the  DoD 
maintenance  environment  is  known  as  "functional  or  performance  testing  1,1  In  the  commercial 


lFunctional  test  is  defined  as  follows:  "A  test  which  determines  whether  the  UUT  is 
functioning  properly.  The  operational  environment  (such  as  stimuli  and  loads)  can  be  either  actual 
or  simulated”  (MIL-STD-1309C).  Functional  testers  gain  access  to  the  UUT  via  the  edge  connector 
of  the  circuit  board,  so  that  the  interconnection  device  is  normally  simple.  Functional  tests  can  be 
either  static  (whereby  UUT  output  measurements  are  made  only  after  they  have  stabilized  with 
respect  to  a  given  input  stimulus,  i.e.,  at  low  speed)  or  dynamic  (whereby  the  testing  sequence  is 
performed  at  the  UUT’s  rated  speed).  The  choice  between  the  two  is  based  on  cost  effectiveness. 
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maintenance  environment,  "in-circuit  testing”  is  more  frequently  used.2  A  mix  of  these  two 
approaches  is  used  in  the  manufacturing  process  with  in-circuit  testers  (in  conjunction  with  other 
quality  control  methods)  applied  at  the  lower  levels  of  indenture  (piece  parts,  circuit  boards)  and 
functional  testers  at  the  higher  levels  of  indenture  (circuit  boards,  assemblies).  Testing  at  the  sub¬ 


system  and  system  levels  in  the  manufacturing  plant  is  generally  a  manual  operation,  using  either 
hot  mockups  or  the  weapons  system  itself  with  its  built-in  test  (BIT).  The  testing  philosophy  has  a 
strong  impact  on  the  cost  of  testers  and  associated  test  programs.  A  rough  comparison  is  shown  in 
Table  4-1.  Manufacturers  are,  therefore,  vitally  interested  in  determining  the  most  cost-effective  mix 
of  automatic  testing  applications  in  their  production  line  as  a  function  of  volume,  fault  spectrum,  and 
yield  at  each  stage  of  production. 

TABLE  4-1.  COMPARISON  OF  AUTOMATIC  TESTING  PHILOSOPHIES 


CRITERIA 

IN-CIRCUIT 

TEST 

FUNCTIONAL 

TEST 

Cost  of  ATE 

Low 

High 

Set-Up  Costs 

•  Programming 

Low 

High 

•  Fixtures 

High 

Low 

Testing  Costs  (go/no-go) 

High 

Low 

Diagnosis  Costs 

Low 

High 

Repair  Costs  of  UUT 

Low 

High 

Fault  Coverage  (affects  inversely  the  next  stage  test  cost) 

Low 

High 

SOURCE:  Brendan  Davis.  The  Economics  of  Automatic  Testing  (London:  McGraw-Hill  Book 
Company,  Ltd.,  1982). 


2In-circuit  test  is  defined  as  follows:  "Tests  of  individual  components  within  a  circuit  while 
guarding  out  the  effects  of  surrounding  components  (analog)  or  overriding  (digital)  inputs”  (MIL- 
STD-1309C)  In-circuit  testers  require  bed-of-nails  fixtures  to  gain  access  to  test  points  on  the 
circuit  board,  such  fixtures  are  complex  and  costly  compared  to  the  interconnection  devices 
normally  required  for  functional  board  testers. 


In  recent  years,  technology  advances  have  weakened  the  traditional  distinctions  outlined 
above.  For  example,  the  commercial  ATE  industry  is  starting  to  combine  the  two  test  approaches  into 
ATE  designed  to  do  both  in-circuit  and  functional  testing.  The  simpler  versions  of  those  testers  use  a 
split  approach,  with  in-circuit  testing  for  analog  circuitry  and  functional  testing  for  digital  circuitry. 
The  more  complex  versions  provide  a  full  in-circuit  testing  and  functional  testing  capability  in  the 
same  mainframe.  However,  their  high  cost  limits  them  to  special-purpose  performance-testing 
applications.  The  use  of  two  separate  testers  has  become  standard  practice,  with  in-circuit  testers 
used  as  a  screening  device  for  manufacturing  defects  or  damage  and  functional  testers  used  for 
evaluating  functional  performance  and  diagnosing  faults.  Another  recent  development  in  the  ATE 
industry  is  the  adoption  of  a  new  testing  philosophy  for  microprocessor  boards.  The  approach,  known 
as  "in-circuit  emulation,”  has  become  popular  because  it  overcomes  the  limitations  of  digital  in- 
circuit  testers  (such  as  the  inability  to  detect  critical  logic  timing  faults,  inability  to  test  circuitry 
interaction,  and  limited  ability  to  perform  dynamic  testing)  without  incurring  the  disadvantages  of 
functional  testers  (including  high  ATE  costs,  high  programming  costs,  and  fault  isolation 
ambiguity).3  With  most  of  today’s  digital  circuit  boards  containing  a  microprocessor,  the  in-circuit 
emulation  test  approach  can  be  expected  to  become  the  standard  test  philosophy  in  the  commercial 
sector  for  field  service  of  digital  boards.  In  the  military  sector,  however,  this  approach  cannot  be 
applied  because  circuit  boards  are  generally  designed  to  military  specifications  that  presently  require 
them  to  be  hard  wired,  thereby  prohibiting  the  use  of  plug-in  microprocessors  4 

Other  trends  in  automatic  testing  are  being  driven  by  technology  advances  in  prime  equipment 
that  result  in  ever-increasing  component  density.  The  industry  adoption  of  "surface  mount 

3For  a  general  description  of  in-circuit  testing  and  in-circuit  emulation  techniques  and 
equipment,  see:  John  Bateson,  "In-Circuit  Testers  for  Service  and  Repair”,  Evaluation  Engi¬ 
neering.  July  1984,  or  the  same  author’s  book,  In-Circuit  Testing  (Florence,  Kentucky:  Van 
Nostrand  Reinhold  Co.,  Inc.,  1985). 

4One  DoD  document  that  advocates  using  sockets  for  complex  integrated  circuits,  is  a  recent 
Air  Force  design  guide:  Directorate  of  Support  Systems  Engineering,  "Onboard  Test  (OBT) 
Improvement  Project  -  Phase  II,”  ASD-TR-83-5012  (Wright-Patterson  Air  Force  Base,  Ohio:  Aero¬ 
nautical  Systems  Division,  Air  Force  Systems  Command,  September  1983) 


technology”  to  permit  higher  component  density  on  a  circuit  board  makes  in-circuit  testing  of  those 
components  impossible  (in-circuit  testing  can  still  be  applied  to  the  buses  and  controls  of  such  boards), 
so  that  functional  testing  will  be  a  requirement,  not  an  option,  in  the  manufacturing  process.  The 
impending  introduction  of  very-high-speed  integrated  circuit  technology  and  the  impact  of  that  tech¬ 
nology  on  ATE  requirements  unknown  at  the  present  time,  but  such  circuits  will  require  functional 
testing.5 

In  sum,  the  traditional  dichotomy  between  automatic  testing  requirements/techniques  in  the 
manufacturing  plant  and  those  for  field  service  is  beginning  to  disappear,  with  a  convergence  to  the 
same  testing  technique  (functional  testing).  Consequently,  the  opportunity  for  integrating,  or  coordi¬ 
nating,  the  development  of  test  programs  required  in  weapons  system  production  and  maintenance  is 
increasing  Exploiting  that  opportunity  would  reduce  the  cost  for  TPSs  and  shorten  the  time  needed 
to  develop  them. 

The  remainder  of  this  chapter  is  focused  on  the  present  TPS  acquisition  and  support  process  in 
the  Military  Services.  That  process  is  totally  divorced  from  that  of  the  prime  and  subcontractors  for 
their  own  automatic  testing  needs  in  the  manufacturing  process. 

INTRODUCTION 

The  acquisition  or  development  of  TPSs  is  a  complex  process.  In  the  past,  little  guidance  was 
available  to  program  (project)  managers  as  an  aid  in  managing  the  process  effectively.  Both  industry 
and  DoD  had  to  cope  with  a  shortage  of  expert  test  program  designers.  Automated  support  tools  had 
to  be  developed  to  improve  the  productivity  of  test  program  designers  and  the  quality  of  test  pro¬ 
grams;  until  the  late  1970’s,  the  absence  or  limited  availability  of  such  tools  translated  into  high  TPS 
development  costs,  especially  those  for  line  replaceable  units  (LRUs).  Because  of  the  high  cost  of  this 
support  element  for  a  new  weapons  system  and  its  late  development,  funding  was  invariably  cut  or 
deferred.  In  the  absence  of  an  effective  evaluation  of  the  "quality”  of  test  programs  (diagnostic 
performance  as  well  as  run  time),  notwithstanding  design  reviews  and  formal  acceptance  tests, 

5For  the  ATE  industry’s  viewpoint  see,  for  example:  "Military  ATE,”  Evaluation  Engi¬ 
neering.  November  1985 


contractor  incentives  were  not  practical  in  TPS  procurement;  instead,  TPSs  were  procured  under 
"fixed-price”  or  "cost-plus”  contracts.  In  most  cases,  these  contracts  were  with  prime  contractors  (to 
minimize  program  management  problems  and  Government  risks  with  integrating  TPSs  and  ATE), 
which  created  more  of  a  disincentive  than  incentive  for  high-quality  TPSs.  These  sole-source 
contracts  for  TPS  development  tended  to  be  costly  because  contractors  tried  to  protect  themselves 
against  the  risks  involved,  especially  the  uncertain  availability  of  Government-furnished  ATE  and 
test  articles  (UUTs)  and  the  uncertainty  caused  by  ambiguous  acceptance  criteria 

Examples  of  recent  exceptions  to  this  long-standing  practice  are  the  Navy’s  competitive 
procurement  of  TPSs  for  the  F/A-18,  resulting  in  a  significant  reduction  in  acquisition  costs  (although 
it  still  remains  to  be  seen  whether  this  approach  will  improve  the  diagnostic  performance  and  main¬ 
tainability  of  the  resulting  TPSs),  and  the  Air  Force’s  recent  practice  to  compete  TPSs  for  shop 
replaceable  units  (with  the  responsible  ALC  frequently  the  winner  based  on  cost).  The  Army  has  also 
routinely  developed  a  portion  of  its  TPSs  for  circuit  boards  in-house,  using  the  organic  capabilities 
developed  at  its  Tobyhanna,  Pennsylvania,  and  Sacramento,  California,  depots. 

Operational  test  programs  consist  of  three  sections.  The  first,  often  referred  to  as  the  "pre¬ 
amble  survey,”  is  designed  to  check  the  availability  and  status  of  the  stimulus  and  measurement 
instruments  that  will  be  invoked  by  the  other  two  sections  of  the  test  program.  It  also  checks  whether 
the  correct  interconnection  device  has  been  mounted;  whether  it  has  been  hooked-up  correctly;  and 
whether  it  is  operable.  The  second,  "go  chain,”  performs  an  end-to-end  functional  test  if  no  faults  are 
present,  but  terminates  with  the  first  discrepancy  detected.  The  third  section,  "no-go  chain,”  has 
multiple  entry  points  (from  the  second  section)  and  is  designed  to  isolate  each  functional  discrepancy 
to  one  or  more  replaceable,  physical  elements  (e  g.,  piece  parts  if  the  L'UT  is  a  circuit  board  or  circuit 
boards  if  the  L'UT  is  a  higher  level  assembly). 

The  last  two  sections,  fault  detection  and  fault  isolation,  can  be  implemented  in  different  ways. 
For  fault  detection,  the  alternatives  are  to  compare  the  measured  UUT  responses  'after  applying 
input  signals  through  the  edge  connectors)  either  to  stored  response  patterns  or  to  the  responses  mea¬ 
sured  from  a  known  good  L’UT  The  former  ("stored  pattern  functional  testing”)  is  the  standard 


approach  for  DoD  test  programs,  whereas  the  latter  ("dynamic  reference  testing”)  is  more  common  in 
the  commercial  sector.  For  fault  isolation,  the  alternatives  are  "fault  dictionary”  and  "computer- 
guided  probing.”  In  the  fault  dictionary  approach,  diagnostic  test  routines  enter  a  large  data  base  and 
perform  additional  tests  as  indicated  to  match  a  given  discrepancy  and  associated  cause(s)  that  have 
been  determined  through  prior  analysis.  Because  of  the  existence  of  "equivalent  faults”  (faults 
responsible  for  the  same  response  pattern  that  cannot  be  distinguished  by  any  set  of  input  patterns), 
the  fault  dictionary  approach  frequently  results  in  an  ambiguous  diagnosis,  with  a  list  of  possible 
causes  of  the  detected  failure  symptom.  In  contrast,  computer-guided  probing  is  based  on  a  fault 
model  of  the  UUT  with  interactive  probing  instructions  displayed  on  the  ATE’s  graphic  display. 
Backtracking  through  the  circuit  along  a  "bad”  path,  the  model  directs  the  operator  to  place  a  logic 
probe  on  selected  test  points.  For  each  point  probed,  it  reruns  the  test  program,  while  monitoring  the 
logic  values  sensed  by  the  probe,  until  a  device  is  found  that  has  all  good  inputs  but  one  or  more  bad 
outputs.  Both  approaches  (fault  dictionary  and  guided  probe)  are  able  to  isolate  only  one  fault  at  a 
time  and  require  the  isolated  fault  to  be  repaired  before  the  process  can  be  restarted  to  detect  and 
isolate  other  possible  faults  in  the  UUT. 

The  Military  Services  have  adopted  the  fault  dictionary  approach  as  the  standard  approach  for 
their  TPSs,  whereas  computer-guided  probing  is  the  preferred  method  in  commercial  applications. 
The  main  advantages  of  the  fault  dictionary  approach  are  the  speed  of  test  program  execution  and  the 
avoidance  of  ATE  operator  intervention  ( i.e. ,  the  approach  is  perceived  as  reducing  the  skill  require¬ 
ments  compared  to  guided  probing).  To  compensate  for  the  resulting  ambiguity  of  fault  isolation, 
some  fielded  TPSs  include  (noninteractive)  probing  instructions  either  on  the  video  display  terminal 
or  in  the  test  program  instruction.  Military  Service  policies  and  practices,  however,  differ  on  this 
issue.  For  example,  the  Army’s  Aviation  Systems  Command  has  a  policy  that  prohibits  the  ATE 
operator  from  accessing  the  inside  of  a  UUT  at  the  intermediate-maintenance  level -a  policy  largely- 
inspired  by  the  limited  training  and  experience  of  ATE  operators  in  the  Army  and  the  need  to 
minimize  inadvertent  LRU  damage.  Nevertheless,  technology  is  driving  the  need  for  probing  within 
a  UUT  in  order  to  reduce  fault  isolation  ambiguity:  the  increasing  device  and  circuit  board 


complexity  impose  testing  requirements  beyond  the  test  point  access  afforded  by  edge  connectors,  so 
that  additional  test  points  must  be  accessed  for  fault  isolation. 


The  answer  to  this  problem  is  to  place  more  emphasis  on  design  for  testability.  The  consensus 
is  that  inadequate  design  for  testability  is  the  principal  cause  of  many  TPS  performance  problems. 
With  the  recent  promulgation  of  the  first  military  standard  on  testability,6  the  expectation  is  that 
design  for  testability  will  receive  more  emphasis  and  scrutiny  in  the  weapons  system  acquisition 
process.  Currently,  TPS  quality  is  measured  in  three  ways: 

•  Fault  Coverage  or  Fault  Detection  Rate.  This  metric  is  defined  as  the  percentage  of 
possible  hardware  failures  detectable  by  the  TPS.  If  the  percentage  is  well  below 
100  percent  (i.e.,  not  detecting  existing  failures)  or  above  100  percent  ( i . e . ,  indicating 
failures  where  none  exists),  the  TPS  loses  much  of  its  practical  value  and  the  user  loses 
confidence  in  test  results. 

•  Fault  Isolation  Resolution  Level.  This  metric  specifies  the  ambiguity  of  automated  fault 

isolation,  commonly  expressed  in  terms  of  a  series  of  percentages  of  detectable  failures 
isolated  to  1,  2,  ...  n  removable  items,  where  n  is  the  maximum  size  of  the  ambiguity 

group.  If  failures  are  isolated  by  the  TPS  to  a  large  ambiguity  group,  manual  trouble¬ 
shooting  is  necessary 

•  R  m  Time.  This  metric  is  the  elapsed  time  for  executing  the  TPS,  normally  measured  for  a 
complete  end-to-end  functional  check  for  a  UUT  without  a  failure. 

TPS  development  contracts  typically  include  quantitative  goals,  not  firm  requirements,  for  the 
first  two  metrics.  The  third  metric  is  normally  not  addressed,  except  for  a  reminder  that  minimum 
run  time  is  desirable.  TPS  run  time  is  counted  in  seconds  for  digital  circuit  boards;  hours  for  complex 
assemblies  such  as  LRUs.  The  two  primary  factors  influencing  both  diagr  ''Stic  performance  and  run 
time  are  design  for  testability  and  L’UT-ATE  compatibility.  Additionally,  run  time  is  very  much 
influenced  by  the  experience  of  the  TPS  developer  as  well  as  the  users  ability  to  modify  fielded  TPSs 
based  on  empirical  data.  For  example,  experience  shows  that  TPS  run  time  frequently  can  be  reduced 
by  30  to  80  percent  through  rearranging  the  sequence  of  test?  performed.  As  another  example,  the 
Air  Force  found  that  oy  using  multiple  entry  points  in  the  functional  test  section  of  TPSs  lor  avionics 


6VIIL-STD-2i65,  "Testability  Program  for  Electronic  Systems  and  Equipment”  (Washington, 
D  C  Naval  Electronic  Systems  Command,  26  January  1985).  This  military  standard  is  "approved 
for  use,"  only;  it  is  not  prescribed 
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LRUs  (combined  with  recording  fault  codes  displayed  by  BIT  in  the  cockpit  display),  it  did  not  have  to 
double  the  ATE  capacity  in  the  F-16  avionics  intermediate  shop  to  support  peacetime  sortie  rates  as 
planned  by  AFSC.  Unfortunately,  such  exploitation  of  BIT  information  at  intermediate-level  mainte¬ 
nance  activities  still  remains  an  exception  today. 

TPS  ACQUISITION 

The  specific  policies  and  procedures  for  TPS  acquisition  differ  among  the  Military  Services,  but 
the  generic  process  consists  of  the  following  series  of  interrelated  steps  that  are  common  to  each: 

•  Test  requirements  specification 

•  Test  requirements  analysis 

•  TPS  development 

•  TPS  integration  and  acceptance  test. 

An  overview  of  the  generic  process  is  provided  by  Figure  4-1,  which  happens  to  apply  to  the 


Air  Force.  A  brief  description  of  each  of  the  above  steps  follows. 

Test  Requirements  Specification 

Development  of  the  UUT  test  specifications  is  part  of  the  design  engineering  and 
technical  documentation  effort;  it  is  not  included  in  the  TPS  development  contract  In  the  absence  of 
a  DoD-wide  standard,  both  the  Air  Force  and  Navy  have  adopted,  independently,  their  own  standards 
for  the  format,  scope,  and  content  of  the  formal  documentation  of  this  effort,  known  as  the  test 
requirements  document  (TRD).7 * * 10  The  Army  has  not  yet  developed  such  a  standard.  Some  of  the  basic 
problems  with  the  timely  specification  of  valid  test  requirements  are  described  in  the  following  sub¬ 
sections. 


7Air  Force:  MIL-STD-1519  (USAF),  "Preparation  of  Test  Requirements  Document,” 

17  September  1971  (with  Notice  l,  1  August  1977). 

Navy:  MIL-STD-1345B  (Navy),  "Test  Requirements  Document,  Preparation  of’: 

10  February  1981,  and  MIL-STD-2076(AS),  "Unit  Under  Test  Compatibility  with  ATE,  General 
Requirements  for”;  l  March  1978. 

Army:  Design  Standard  for  the  Development  of  AN/USM-410  Test  Program  Sets.  RCA 
Document  CR-82-588-04,  December  1982  (revised  February  1984).  This  Army  standard  does  not 
use  the  term  TRD,  but  directs  that  "test  specifications”  be  included  in  the  "UUT  data  package,” 
whenever  possible. 


TPS  DEVELOPMENT  AND  MAINTENANCE  PROCESS  (AIR  FORCE)  (CONTINUED 


Tailoring.  The  preparation  instructions  in  MIL-STD-1519  ensure  that  the  resulting 
TRD  provides  a  test  program  source  documentation  baseline  (independent  of  the  ATE  on  which  the 
TPS  is  to  be  implemented)  that  can  be  used  for  the  acquisition  of  a  test  program  and  its  life  cycle 
support  (maintenance).  Development  of  a  TRD,  however,  is  a  laborious,  tedious,  time  consuming,  and 
expensive  process.  Furthermore,  (JUT  configuration  changes  from  preproduction  prototypes  to  pro¬ 
duction  baseline  require  TRD  updates,  and  program  schedules  often  require  starting  with  TPS 
development  before  a  complete  TRD  can  be  prepared.  As  a  result,  most  contracts  that  require 
delivery  of  TRDs  have  been  tailored  to  the  specific  needs  of  each  program.  For  example,  during  full- 
scale  development,  TRDs  may  be  limited  to  functional,  end-to-end  tests  of  major  assemblies,  with 
subsequent  expansion  into  fault  diagnostics  at  the  component  level  after  the  program  transitions  into 
production.  If  the  prime  contractor  is  also  to  produce  the  TPS,  the  TRD  to  be  delivered  may  be  limited 
to  only  those  data  required  for  configuration  control  and  TPS  verification/validation  testing.  Alterna¬ 
tively,  earlier  in  the  program,  limited  TRDs  may  be  contracted  solely  to  acquire  data  necessary  for 
ATE  selection  or  maintenance  concept  evaluation  For  many  of  the  currently  fielded  TPSs,  a 
complete  TRD  has  never  been  acquired.  The  absence  of  a  TRD  makes  organic  maintenance  of  the 
associated  TPS  very  difficult  although  not  necessarily  impossible  as  long  as  UUT  technical  data  are 
available  in  other  forms 

Verification/ Validation  The  TRD  is  supposed  to  provide  a  complete  specification  of  all 
tests  required  to  ascertain  that  the  UUT  is  operating  within  design  performance  parameters,  includ¬ 
ing  detailed  specifications  of  UUT  input  conditions,  output  measurements,  and  digital  patterns  All 
three  TRD  military  standards  include  comprehensive  UUT  descriptions,  drawings,  wiring  diagrams, 
and  information  on  theory  of  operations  and  operational  software  The  Navy  version  also  includes 
ATE  information  and  ATLAS  (Abbreviated  Test  Language  for  All  Systems)  statements  corre¬ 
sponding  to  the  English  language  test  descriptions  While  each  of  the  standards  requires  contractor 
delivery  of  a  validation  certification  to  the  procuring  activity,  the  specific  quality  assurance 
provisions  differ  MIL-STD-1519  requires  the  supplier  to  apply  the  inputs  and  loads  specified  in  the 
TRD  to  a  certified  good  UUT  and  verify  that  the  specified  parameter  values  are  obtained  —  a  process  to 


be  witnessed  by  a  procuring  activity  representative.  If  the  UUT  is  not  available,  however,  this 
requirement  is,  of  course,  waived.  MIL-STD-2076  requires  the  supplier  to  deliver  a  "verified  UUT 
performance  characteristic/test  requirements  matrix,"  and  specifies  that  the  procuring  activity 
"reserves  the  right  to  witness  any  of  the  inspections/validations  specified  herein.”  In  summary,  the 
only  tools  are  that  DoD  has  for  verifying  completeness,  accuracy,  and  validity  of  a  TRD  empirical 
evidence,  and  even  that  evidence  is  often  not  available  in  the  absence  of  UUT  hardware.  Thus,  the 
DoD  ultimately  must  rely  on  the  professional  expertise  of  the  contractor  in  procuring  a  TRD 

Automated  Tools.  Contractors  have  developed  various  types  of  automated  tools  to 
reduce  the  amount  of  tedious,  repetitive,  manual  labor  involved  in  generating  a  TRD.  Most  appear  to 
be  pure  automation  approaches,  but  more  sophisticated  systems  have  been  designed  to  consolidate  all 
activities  for  TPS  development  into  a  single  "programming  environment,"  including  generation  of 
TRDs,  ATLAS  test  procedures,  target  ATE  test  programs,  test  program  instructions,  and  inter¬ 
connection  device  design.  Only  a  few  developers,  however,  have  recognized  the  applicability  of  other 
engineering  tools,  such  as  the  Failure  Mode  and  Effects  Analysis  (FMEA)  conducted  by  design 
engineers  in  proving  reliability  performance  and  assessing  BIT  requirements  to  TRD  generation  and 
verification  .8 

Comments.  Although  the  TRD  is  the  preferred  format  for  providing  UUT  technical  data 
including  test  requirements,  it  often  is  not  available  when  needed.  In  addition,  the  TRD  reflects 
factory  quality-assurance  tolerances,  which  frequently  differ  from  operational  tolerances  If  these 
differences  are  not  recognized  in  subsequent  steps  in  TPS  development,  then  the  fielded  TPSs  will 
have  a  high  rate  of  Type  I  errors  (declaring  a  component  bad  when  it  is  actually  operational)  These 
and  other  problems  may  explain  why  the  Navy  and  Army  are  deemphasizing  the  importance  of  the 
TRD.  The  Air  Force,  however,  continues  to  stress  the  importance  of  the  TRD  as  a  necessary,  if  not 

^The  Rome  Air  Development  Center's  current  research  and  development  program  includes  a 
project  entitled  "Automated  Test  Requirement  Document  Generation,”  for  which  recently  a  con 
tract  was  awarded  with  planned  completion  in  mid- 1987  It  has  also  sponsored  a  number  of  studies 
of  automated  FMEA  techniques  and  it  is  planning  to  evaluate  the  use  of  such  techniques  for  TRD 
verification 


sufficient,  condition  for  a  good  TPS.  Essentially,  the  difference  between  the  Air  Force  and  Navy  is 
that  the  Air  Force  treats  the  TRD  as  a  "living”  document  to  be  updated  as  engineering  changes  are 
made  in  the  associated  UUT.  In  contrast,  the  Navy  believes  this  approach  is  not  affordable  and  not 
cost  effective;  it  treats  the  TRD  as  a  "static”  document  and  uses  a  less-detailed  document  (described  in 
the  next  subsection)  to  keep  test  requirements  data  current.  The  viewpoint  of  TPS  developers  is  best 
summarized  in  the  following  statement:  "TRD  misinformation  has  been  proven,  time  and  time  again, 
to  be  a  backbreaker  of  TPS  development.”9 

Test  Requirements  Analysis 

The  next  step  consists  of  a  comprehensive  analysis  of  UUT  source  data  (preferably  in  the 
form  of  a  TRD),  ATE  documentation,  TPS  design  policy  (if  any),  and  the  subsystem/UUT  mainte 
nance  concept  The  purpose  of  this  analysis  is  to  determine  the  exact  stimulus  and  measurement 
criteria  to  be  used  in  the  test  program  and  the  ATE/UUT  interface  impedance-matching  and  signal¬ 
conditioning  requirements  The  results  are  documented  in  a  Test  Strategy  Report  (TSR)  10  As 
indicated  earlier,  the  Army  and  Navy  place  more  emphasis  on  the  TSR  than  on  the  TRD;  considerable 
overlap  exists  between  the  two,  but  the  nature  of  each  document  is  different  The  TSR  contains  the 
following  material: 

•  UUT  functional  description 

•  UUT  operating  mode  description 

•  Test  concept,  consisting  of  a  step-by-step  breakdown  of  the  stimulus  requirements 
and  measurement  parameters  and  accuracies  for  each  functional  test  in  the  perfor¬ 
mance  test  sequence  as  well  as  the  general  fault-isolation  approach 


9Honeywell  Systems  Research  Center,  Unit  Under  Test  Simulator  Feasibility  Study. 
AFWAL-TR-80- 1091  (Wright- Patterson  Air  Force  Base  AFSC/Aeronautical  Systems  Division/ 
Avionics  Laboratory,  June  1980) 

10Content  and  general  format  of  the  TSR  are  specified  in  MIL-STD-20771  AS),  "Test  Program 
Sets,  General  Requirements  for”,  9  March  1978.  This  standard  is  also  invoked  by  the  Army's  TPS 
specification  with  regard  to  TSR  preparation  but  not  by  that  of  the  Air  Force  s.  The  Air  Force  does 
not  recognize  the  term  TSR  nor  the  concept  of  a  TSR  as  a  separate  deliverable  in  addition  to  the 

TRD 


•  Test  constraints,  consisting  of  specification  of  any  incompatibilities  between  UUT 
and  ATE  and  the  resulting  interface  requirements 

•  Test  summary,  consisting  of  descriptions  of  the  power,  sync,  and  signal  inputs 
required  by  the  UUT  and  corresponding  outputs  to  be  monitored  by  the  ATE 

•  Operation  interaction 

•  Run  time  prediction. 

The  actual  or  potential  problems  with  the  TSR  are  similar  to  those  listed  for  the  TRD 
Specifically,  DoD  has  not  developed  tools  to  aid  in  evaluating  the  completeness  and  validity  of  the 
tests  documented  in  the  TSR;  instead,  it  must  rely  on  the  thoroughness  of  design  reviews  conducted 
by,  or  on  behalf  of,  the  procuring  activity.  Another  problem,  especially  when  TPSs  are  procured 
competitively  or  developed  under  little  interaction  with  the  UUT  designer,  is  that  the  test  strategy 
may  be  based  on  factory  acceptance  test  procedures  and  circuit  analyses  rather  than  functional  test¬ 
ing  in  the  operational  environment.  Such  a  strategy,  however,  can  result  in  numerous  testing 
problems  (Type  I  errors)  and  excessive  TPS  run  times.11  Alternatively,  DoD  appears  in  need  of  a 
strategy  that  reflects  how  the  UUT  performs  in  the  next  higher  assembly  The  elements  of  that 
strategy  should  include: 

•  Implementation  of  the  UUT’s  BIT 

•  Functional  testing  in  lieu  of  parametric  testing,  including  simultaneous  application 
of  input  signals  simulating  the  operational  environment  and  use  of  test  tolerances 
representative  of  the  UUT  mission  requirements. 

Such  a  strategy,  apparently,  has  seldom  been  adopted  in  the  past. 

TPS  Development 

The  next  steps  in  TPS  acquisition  are  the  design  and  documentation  of  the  TPS, 
including  test  program,  test  program  instruction,  and  interface  device(s).  A  variety  of  aids  were 
developed  in  the  1970’s  to  support  TPS  development,  including  automatic  test  program  generators 
(ATPGs)  for  digital  UUTs.  The  TPS  design  is  usually  documented  in  the  form  of  functional  flow¬ 
charts  and  diagnostic  flowcharts.  The  test  sequence  normally  consists  of  (1)  identity  checks,  to 

nSee,  for  example,  J  Luis  Hernandez  and  R  Glenn  Wright,  "Test  Program  Runtime  in 
Today’s  ATE  Environment,”  AUTQTESTCON  '83. 
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verify  that  the  proper  L'L'T  and  interface  device  have  been  connected  correctly  to  the  ATE;  (2)  survey 
or  self-tests,  to  verify  that  ATE  functions  required  for  executing  the  test  program  are  operational  and 
to  localize  any  detected  ATE  faults;  (3)  safe-to-turn-on  tests,  to  detect  any  short  circuits  that  might 
damage  the  UUT,  interface  device,  or  ATE  when  power  is  applied  to  the  UUT;  (4)  static  or  dynamic 
performance  tests,  to  check  UUT  performance  under  static  or  dynamic  operating  conditions  in  accor¬ 
dance  with  the  TSR;  (5)  fault-isolation  tests,  to  isolate  detected  UUT  faults  to  a  specified  replacement 
level.  Some  of  the  basic  problems  with  test  program  design  are  described  in  the  following  subsections. 

Simplifying  Assumptions.  It  is  not  uncommon  for  the  developer  to  make  several  simpli¬ 
fying  assumptions  when  designing  the  TPS.12  These  assumptions  serve  to  limit  fault  detection  to 
single,  "hard”  failures  occurring  within  electronic  components.  In  addition,  failures  caused  by  poor 
workmanship,  basic  design  deficiencies,  incorrect  factory  assembly,  accidental  damage,  UUT  canni¬ 
balization,  or  combat  damage  are  normally  excepted  from  the  TPS  design  logic.  As  a  result,  certain 
failures  encountered  in  the  operational  environment  (i.e. ,  failures  in  wiring  or  interconnectors,  multi¬ 
ple  failures,  intermittent  failures  in  digital  modules,  components  damaged  by  poor  handling,  or 
faulty  modules  as  a  result  of  inadequate  quality  assurance)  invariably  cause  ambiguous  fault 
detection  or  isolation  results.  These  types  of  failures,  however,  may  be  as  frequent  as  the  types  of 
failures  which  the  TPS  is  designed  to  detect.  For  example,  in  the  case  of  modern  digital  avionics, 
studies  have  indicated  that  about  50  percent  of  the  failures  encountered  in  operational  use  do  not 
satisfy  the  TPS  design  assumptions. 


12See,  for  example,  Program  Design  Handbook  for  Automatic  Test  Equipment.  (Washington, 
DC:  Naval  Air  Systems  Command,  August  1979).  This  handbook,  describing  the  entire  TPS 
development  process,  was  prepared  through  the  cooperative  efforts  of  RCA,  Grumman  Aerospace 
Corporation,  PRD  Electronics,  AAI  Corporation,  Vought  Corporation,  and  AMG  Associates 

13OSD-IDA  R&M  Study  Report.  "Steps  Toward  Improving  the  Materiel  Readiness  Posture  of 
the  Department  of  Defense,”  30  vols.,  (Alexandria,  Virginia:  Institute  for  Defense  Analyses, 
August  1983).  Specifically,  Volume  IV,  Technology  Steering  Group  Report,  Appendix  III,  provides 
data  on  the  percentage  of  avionics  failures  attributed  to  cables  and  connectors,  summarizing  the 
more  detailed  data  reported  in  Cabling  and  Connectors  Technology.  IDA  Document  D-29  This 
percentage  is  estimated  to  range  from  5  to  50  percent  depending  on  age  of  aircraft,  while  others 
claim  it  may  be  as  high  as  60  percent.  (This  reference  actually  pertains  to  BIT  software,  but  similar 
arguments  apply  to  off-line  test  programs  for  LRUs  I 
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Test  Tolerance  Limits.  Establishing  proper  test  limits  for  static  and  dynamic  tests  is 
ultimately  a  compromise  between  passing  defective  UUTs  (Type  II  errors)  and  failing  good  UUTs 
(Type  I  errors).  In  practice,  it  is  impossible  to  set  test  tolerance  limits  such  that  both  types  of  errors 
are  eliminated.  Three  factors  must  be  considered  in  establishing  test  limits  in  TPSs  for  fault  detec¬ 
tion  as  well  as  fault  isolation:  (1)  upper  and  lower  tolerance  bounds  on  UUT  output  and  circuit  test 
point  signals,  (2)  the  "cone  of  tolerance”  concept,  and  (3)  test  instrument  measurement  and  stimulus 
accuracies. 

Different  techniques  are  employed  for  defining  upper/lower  tolerance  bounds  on  UUT 
signals  under  normal  operating  conditions,  including  worst-case  analysis,  statistical  circuit  analysis, 
and  empirical  techniques.  While  worst-case  design  analysis  is  a  reliable  approach  guaranteeing  a 
100-percent  yield  during  any  performance  test  ( i.e. ,  a  UUT  manufactured  in  accordance  with  such  a 
design  is  virtually  guaranteed  to  pass  the  performance  test),  the  approach  may  not  be  appropriate  for 
in-service  testing  because  defective  UUTs  can  pass  the  test.  A  statistical  analysis  approach  (com¬ 
monly  a  root-sum-of-squares  analysis)  is  more  difficult  to  perform,  but  places  tighter  tolerances  on 
the  UUT  output  signals,  resulting  in  elimination  of  Type  II  errors  at  the  expense  of  slightly  lower 
yield  (theoretically,  99.7  percent)  and  possibility  of  Type  I  errors.  To  uchieve/sustain  maximum 
mission  capability,  weapons  system  maintenance  should  minimize  the  occurrence  of  Type  II  errors 
even  at  the  expense  of  some  affordable  level  of  Type  I  errors.  Where  current  TPS  guidelines  suggest 
worst-case  analysis  for  determining  tolerance  limits,  they  should  be  revised  to  emphasize  the  use  of 
statistical  analysis  whenever  feasible.14 

The  second  factor,  the  tolerance  cone  concept,  refers  to  the  vertical  relationships  among 
tolerance  values  at  different  levels  of  testing  (both  maintenance  echelon  and  equipment  breakdown 


14For  a  numerical  example  illustrating  the  benefits  of  the  statistical  analysis  approach  over 
worst-case  analysis,  see  Charles  Hafer,  "Establishing  Realistic  Test  Tolerance  Limits  for  Test 
Program  Sets,”  AUTOTESTCON  ’83.  pp.  109-117  This  advice  is  in  contrast  to  that  offered  in  the 
Navy’s  TPS  Design  Manual,  which  recommends  the  use  of  worst-case  analysis  for  test  limit 
selection  [see:  Program  Design  Handbook  for  Automatic  Test  Equipment.  (Washington.  DC. 
Naval  Air  Systems  Command,  August  1979)  p  F-4). 
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structure),  with  the  tightest  tolerances  at  the  factory  and  piece-part  levels.  Ignoring  the  tolerance 
cone  in  defining  tolerance  limits  invariably  causes  testing  errors.  Past  experience,  for  example, 
shows  that  this  factor  has  been  a  major  contributor  to  the  "cannot  duplicate,”  "retest  okay,”  and 
"failed  system  check”  problems  encountered  in  avionics  maintenance. 

The  third  factor,  test  instrument  measurement/stimulus  accuracy,  refers  to  the  close¬ 
ness  of  a  measurement  to  its  true  value  as  fixed  by  a  universally  accepted  standard  In  the 
development  of  TPSs,  it  is  not  uncommon  to  specify  an  accuracy  ratio  (defined  as  the  ratio  of  UUT 
tolerance  to  test  instrument  accuracy)  of  at  least  3:1  or  4:1  for  each  programmed  go/no-go 
performance  test.  When  this  ratio  cannot  be  achieved  by  available  test  procedures  (e  g.,  when  the 
standard  ATE  that  must  be  used  lacks  the  required  test  accuracy),  test  limits  must  be  widened  from 
the  analytically  determined  tolerance  limits  or  ATE  capabilities  must  be  augmented  (either  by  the 
ATE  itself  or  by  use  of  an  active  interface  device).  One  problem  is  that  both  accuracy  (as  defined 
above)  and  resolution  (defined  as  the  smallest  change  in  a  signal  attribute  being  measured  that  can 
be  unambiguously  detected  in  a  measurement  process)  of  test  instruments  are  often  degraded  in  a 
complex  ATE  system  by  the  system  environment  and/or  improper  calibration  procedures  and 
practices  15 

Failure  Priorities.  Failure  data  represent  an  essential  input  to  TPS  development  and 
are  normally  included  in  the  TRD  for  the  (JUT  These  data  are  typically  based  on  the  prediction 
models  of  MIL-HDBK  217  because  virtually  all  DoD  contracts  for  military  electronics  require  use  of 
this  handbook  as  the  source  of  reliability  estimates.  Failure  mode  and  failure  rate  data  are  needed  to 
determine.  ( 1 )  the  faults  to  be  addressed  by  the  TPS  such  that  the  contractual  fault-detection  ra  .an 
be  achieved:  (21  the  most  efficient  sequence  of  go/no-go  tests  in  the  UUT  functional  performance  (fault 


lsFor  a  brief  description  of  factors  influencing  test  instrument  degradation  in  the  ATE  sys¬ 
tem  environment,  see:  Maynard  D  Lay  "ATS  Measurements  Today  — How  Good  Are  They9” 
AUTOTESTCON  81.  and  Robert  Buckley,  "Commonly  Misunderstood  ATE  Instrument  Specifi¬ 
cations,”  AUTOTESTCON  ~81  For  a  description  of  the  effects  of  ATE  miscalibration,  see  the  follow¬ 
ing  study  pertaining  to  the  Army's  EQUATE  system  Automatic  Test  Equipment  Calibration/ 
Performance  Verification  Evaluation  and  Research  Program.  NBSIR  82  2601,  (Washington,  D  C 
U  S  Department  of  Commerce,  December  1982)  (For  Official  Use  Only) 
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detection)  test,  with  items  possessing  a  higher  probability  of  failure  placed  first  in  order  to  minimize 
TPS  execution  time;  (3)  cost-effective  tradeoffs  between  fault  isolation  resolution  and  failure 


frequency,  thereby  avoiding  complex  fault  isolation  logic  for  failures  the  .  seldom  occur;  and  (4)  fault 
isolation  thoroughness  and  diagnostic  ambiguity  in  accordance  with  contract  specifications.  As 
indicated  earlier,  we  believe  that  the  entire  process  would  be  improved  substantially  by  a  thorough 
Failure  Mode  Evaluation  Analysis,  but  this  rarely  occurs  due  to  the  time  and  cost  involved  16 

The  chief  problem  with  the  available  failure  data  for  L'UTs  of  a  new  weapons  system  is 
the  lack  of  validity  of  the  engineering  estimates.  Numerous  studies  have  demonstrated  wide  discrep¬ 
ancies  between  predicted  reliability  and  that  achieved  in  the  operational  environment.  For  example, 
the  ratio  of  predicted-to-operational  reliability  may  reach  as  high  as  20:1  for  ground  as  well  as 
airborne  electronics.  While  there  is  some  controversy  regarding  the  implications  of  this  empirical 
evidence  ( i.e. ,  the  need  for  either  better  reliability  models  or  better  quality  assurance/control  of  the 
manufacturing  process17),  the  implication  for  TPSs  is  clear;  failure  data  feedback  from  the  field 
environment  is  essential  to  improve  the  run  time  and  diagnostic  performance  of  fielded  TPSs  because 
their  design  assumptions  are,  by  necessity,  invalid.  In  those  cases  in  which  this  type  of  feedback  has 
occurred,  TPS  run  times  have  been  reduced  by  50  percent  or  more. 

Automatic  Test  Program  Generators  The  general  structure  of  an  ATPG  is  illustrated  in 
Figure  4-2  The  first  phase  consists  of  generating  a  model  of  the  UUT,  based  on  logic  diagram  input 
specifications.  The  ATPGs  that  are  currently  available  differ  from  each  other  in  terms  of  level  of 
modeling  capacity  and  availability  of  integrated  circuit  libraries  Output  from  this  phase  includes  a 


lfiFor  a  brief  description  of  alternative  FMEA  techniques  and  the  implementation  of  a 
Government-owned,  standardized,  automated  FMEA  system  (completed  in  1984),  see  Heather  R 
Dussault,  "Automated  FMEA  —  Status  and  Future."  1984  Proceedings  Annual  Reliability  and 
Maintainability  Symposium. 

■"Compare,  for  example,  the  following  two  papers  Kam  L  Wong,  "L'nified  Field  (Failure) 
Theory  —  Demise  of  the  Bathtub  Curve,’’  1981  Proceedings  Annual  Reliability  and  Maintainability 
Symposium,  and  Clifford  M  Ryerson,  "The  Reliability  Bathtub  Curve  is  Vigorously  Alive.”  1982 
Proceedings  Annual  Reliability  and  Maintainability  Symposium 


FIGURE  4-2.  AUTOMATIC  TEST  PROGRAM  GENERATION  PROCESS 
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topology  data  base  (defining  all  UUT  components  and  their  interconnections)  and  an  input/output  pin 
list  for  subsequent  use  by  the  system. 

The  second  phase  consists  of  an  interactive  loop  comprising  stimulus  pattern  generation 
(either  provided  automatically  by  the  system  or  input  by  the  user  in  an  interactive  ATPG  process), 
good  circuit  simulation,  and  fault  simulation.  The  good  circuit  simulation  process  determines  the 
UUT  model  responses  to  the  generated  stimulus  pattern,  creating  data  files  on  the  expected  responses 
~t  the  output  pins  for  each  stimulus  pattern  (the  response  data  in  combination  with  the  stimulus  data 
constitute  the  go/no-go  tests)  and  the  model  response  data  (which,  in  combination  with  the  topology 
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data  base,  provide  the  information  for  guided  probe  diagnostic  testing).  The  fault-simulation  process 
determines  the  UUT  model  responses  to  the  generated  stimulus  pattern  in  the  presence  of  faults, 
creating  a  fault  dictionary  data  base,  a  listing  of  undetected  faults,  and  metrics  of  fault  coverage 
achieved.  The  available  ATPGs  differ  in  the  methodology  of  stimulus  pattern  generation  (the  most 
effective  and  frequently  used  method  is  known  as  "path  sensitization,”)18  level  of  simulation  accuracy 
(number  of  logic  states  and  timing  simulation),  and  fault-simulation  approach  (deductive,  parallel,  or 
concurrent). 

The  third  and  final  phase  of  the  ATPG  process  consists  of  a  postprocessor  to  transform 
the  ATPG  output  data,  in  combination  with  ATE  descriptive  data,  into  a  test  program  tailored  to,  and 
in  the  language  of,  the  target  ATE  system. 

While  ATPGs  are  an  important,  if  not  essential,  tool  for  developing  test  programs  for 
digital  UL'Ts,  some  common  misperceptions  exist  about  their  use  and  effectiveness.  First,  all  ATPGs 
require  a  considerable  extent  of  user  interaction,  especially  when  the  UUT  is  a  higher  level  assembly. 
Thus,  they  do  not  reduce  the  skills  required  for  producing  effective  test  programs  compared  to  those 
required  for  a  completely  manual  process;  their  chief  benefit  is  the  increase  in  productivity  of  test 
program  engineers  or  programmers.  Second,  even  the  best  available  ATPG  is  subject  to  a  number  of 
limitations  that  restrict  the  diagnostic  performance  of  the  resulting  TPS.  Among  these  limitations, 
the  following  three  are  worthy  of  note. 

All  ATPGs  are  limited  to  certain  types  of  faults  (primarily,  "stuck  at”  faults  of  some  also 
model  such  fault  types  as  shorts  between  adjacent  pins  and  selected  integrated  circuit  faults).  None, 
however,  simulates  pattern-sensitive  faults  (a  failure  mode  prevalent  in  tightly  packed  components 
and  caused  by  interference  between  logic  values  under  certain  input  patterns)  nor  certain  physical 
fault  modes,  such  as  shorts  occurring  in  feedback  loops,  for  which  "stuck-at”  models  are  inadequate. 19 

18For  a  concise  description  of  the  path  sensitization  method,  see:  John  Grason  and 
Andrew  W.  Nagle,  "Digital  Test  Generation  and  Design  for  Testability,"  Proceedings  17th  Design 
Automation  Conference.  June  1980. 


Another  shortcoming  is  that  ATPGs  do  not  account  for  the  differential  reliability  of  the 
nodes  of  the  UUT.  They  consider  all  nodes  to  be  equally  likely  to  fail,  given  a  UUT  failure  Because 
this  assumption  is  invalid,  the  consequence  is  that  the  reported  fault  coverage  I  fault-detection  rate)  is 
invalid;  the  percent  of  failures  (weighted  by  failure  frequency)  actually  detected  and  isolated  by  the 
TPS  may  be  much  less  than  expected  and  advertised.  20 

Yet  another  shortcoming  pertains  to  those  ATPGs  that  model  UUT  devices  such  as 
integrated  circuits  at  the  functional  rather  than  the  gate  level.  Only  a  handful  of  the  present  ATPGs 
incorporate  a  gate-level  modeling  approach.  Yet,  several  investigations  have  shown  the  inadequacy 
of  functional-level  models  in  detecting  integrated  circuit  internal  faults  that  affect  UUT  performance 
For  example,  a  comparative  evaluation  sponsored  by  the  F-16  Program  Office  showed  that  TPSs 
developed  by  using  functional-level  ATPGs  could  detect  only  60  percent  of  integrated  circuit  internal 
faults,  while  those  developed  with  the  aid  of  gate-level  ATPGs  detected  98  percent.21  This  difference 
in  detection  rates  caused  the  F-16  Program  Office  to  prescribe  use  of  the  best  available  gate-level 
ATPG,  known  as  LASAR  (Logic  Automated  Stimulus  and  Response),  in  contracting  for  F-16  TPSs. 22 
(NAVAIR  has  also  mandated  use  of  LASAR  in  the  development  of  test  programs  for  all  N'AVAIR  ATE 
applications.  Similarly,  the  Army’s  TPS  specification  advocates  the  use  of  a  particular  version  of 
LASAR,  known  as  D-LASAR,  whenever  possible.)  The  consequence  of  using  an  inadequate  ATPG  is 
false  confidence  in  test  comprehensiveness,  leaving  the  user  with  the  incorrect  assumption  that 
patterns  that  detect  integrated  circuit-pin  faults  will  also  detect  related  internal  faults  and  causing 
costly  problems  (Type  II  errors)  in  field-level  maintenance 


20This  deficiency  can  be  corrected  by  combining  an  FMEA  with  the  ATPG,  as  described  by: 
Samiha  Mourad,  "An  Optimized  ATPG,”  Proceedings  17th  Design  Automation  Conference. 


June  1980.  This  approach,  however,  is  not  a  known  practice  in  TPS  acquisition  by  the  DoD 

2>Bruce  A.  Pomeroy,  "LASAR:  Quality  Assurance  for  Digital  Field  Testing," 
AUTOTESTCON  79 

22Many  versions  of  LASAR  evolved  in  the  1970’s  from  the  original  "D-2  LASAR”  that  was 
procured  by  the  Navy  The  most  powerful  version  currently  available  is  LASAR  Version  6  which  is 
proprietary  to  Teradyne,  Inc  This  system  has  been  installed  at  approximately  150  sites  within  the 
United  States,  about  60  percent  of  which  are  defense  related  (defense  contractors  and  military 
organizations) 


The  third  misperception  about  ATPG  effectiveness  is  caused  by  the  technology  trend  in 
prime  equipment  toward  increased  integration.  While  some  of  the  ATPGs  have  been  useful  and  effec¬ 
tive  aids  in  the  past,  the  technology  trend  (exemplified  by  large-scale  integration  and  very-large-scale 
integration,  distributed  microprocessors,  etc.)  makes  all  current  ATPGs  obsolete,  so  that  more 
powerful  aids  must  be  developed.  NAVAIR  has  sponsored  development  of  a  more  flexible  ATPG 
designed  to  cope  with  the  large-  and  very-large-scale  integration  complexity.  That  system,  known  as 
Hierarchical  Integrated  Test  Simulator,  became  the  new  standard  ATPG  for  Navy  avionics  in  1984, 
with  strict  configuration  control  maintained  by  a  software  configuration  control  board  and  limited 
distribution  of  source  code  to  other  DoD  organizations.  The  advantages  of  a  Government-owned 
standard  ATPG,  however,  will  not  be  realized  if  the  Navy  is  not  prepared  to  invest  the  necessary 
resources  to  support  continued  development,  enhancement,  and  maintenance  of  the  Hierarchical 
Integrated  Test  Simulator 

Software  Language  Historically,  TPSs  have  been  programmed  in  many  different  lan¬ 
guages.  In  the  1970’s,  industry  began  to  move  toward  adopting  ATLAS  as  the  standard  test  program 
language.  In  1976,  the  first  industry-wide  standard  for  ATLAS  was  released  and  approved  by  OSD  as 
an  interim  standard  language  for  ATE.23  By  1980,  ATLAS  was  close  to  universal  adoption  by 
industry,  with  test  programs  typically  falling  into  three  categories:  standard  ATLAS,  ATLAS  subset 
(no  extensions  to  standard  ATLAS,  but  excluding  some  vocabulary,  statements,  or  syntactic  options), 
and  adapted  ATLAS  (modified  for  a  particular  test  environment  and  not  controlled  by  the  ATLAS 
standard).  To  promote  further  standardization,  an  international  subset  was  prepared  in  1981, 
approved  by  the  Institute  of  Electrical  and  Electronics  Engineers,  Inc.  (IEEE)  ATLAS  Committee  and 

23For  a  brief  history  of  ATLAS  development,  starting  with  the  work  by  the  Airlines 
Electronic  Engineering  Committee  (the  ARINC  Committee  that  originated  ATLAS)  in  1965,  see: 
IEEE  Guide  to  the  Use  of  ATLAS.  IEEE  Standard  771-1980  (New  York  The  Institute  of  Electrical 
and  Electronic  Engineers,  Inc.,  1980)  DoD  policy  was  first  promulgated  in  a  1976  letter  by  the 
Assistant  Secretary  of  Defense  (Installations  and  Logistics)  directing  convergence  of  the  two 
standard  test  languages  at  the  time,  ATLAS  and  OPAL  (Operational  Performance  Analysis 
Language),  to  be  replaced  by  a  single  test  language  as  of  January  1979,  with  the  Navy  designated 
the  lead  Military  Service  in  the  ATLAS  standardization  effort. 


submitted  to  the  IEEE  Standards  Board,  and  approved  for  publication  as  IEEE  Standard  716, 
Common  ATLAS  (C/ATLAS),  in  early  1982  (with  syntax  diagrams  published  separately  in  IEEE 
Standard  717).  Since  1981,  the  Military  Services  have  designated  C/ ATLAS  IEEE  Standard  716- 
1982  as  the  standard  test  language,  so  that  a  standard  compiler  can  now  serve  multiple  applications. 
Both  standards,  IEEE  Standard  416  and  IEEE  Standard  716,  are  reviewed  and  updated  annually 
under  the  auspices  of  the  "Standards  Coordinating  Committee  for  ATLAS  and  Related  Standards,” 
which  is  known  also  as  the  IEEE  ATLAS  Committee.  That  committee  also  oversees  the  Users  Guide 
Subcommittee  (IEEE  Standard  771)  and  subcommittees  charged  with  developing  three  new  draft 
standards:  test  equipment  description  language,  UUT  description  language,  and  ATPG  output  data 
format. 

Most  of  the  fielded  TPSs  have  been  written  in  some  form  of  adapted  ATLAS  or  in  other 
languages,  which  causes  problems  when  they  must  be  updated  because  of  CUT  changes  or  when  they 
must  be  transferred  to  a  different  ATE  because  of  ATE  obsolescence.  These  problems,  however,  will 
gradually  disappear  as  C/ATLAS  becomes  more  widely  implemented. 

Another  problem,  which  is  inherent  to  any  language  standardization  effort,  is  that  the 
vocabulary  does  not  meet  all  testing  requirements.  ATLAS  is  thus  designed  to  permit  use  of  non- 
ATLAS  sections  in  a  test  program.  With  further  enhancements  to  ATLAS,  the  expectation  is  that  the 
need  for  such  nonstandard  extensions  can  be  minimized  Since  the  first  issue  of  ARINC  Specification 
416  (October  1968,  for  analog  tests  only),  ATLAS  has  already  grown  from  34  verbs,  30  nouns,  and 
200  other  words  to  over  500  characters  and  keywords  in  level  21  (published  in  1984) 

TPS  Integration  and  Acceptance  Test 

The  final  steps  in  the  TPS  acquisition  process  are  known  as  "integration”  and  "accep 
tance  test.”  Integration  refers  to  exercising  the  TPS  on  the  ATE  with  the  UUT,  interface  device, 
cables,  and  documentation  in  order  to  validate  and  verify  TPS  performance.  This  is  normally  per¬ 
formed  in  three  phases:  (1)  forcing  the  TPS  through  its  functional  go-path  tests,  using  a  known  good 
UUT:  (2)  forcing  no-go  paths;  and  (3)  physically  inserting  failures  in  the  UUT.  After  the  test 
program  has  been  debugged  and  integration  successfully  completed,  TPS  performance  is 
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demonstrated  in  a  final  acceptance  test  procedure  in  accordance  with  a  contractor-prepared, 
Government-approved  acceptance  test  plan.  The  overall  process,  i.e.,  integration  and  acceptance 
testing,  typically  accounts  for  25  to  55  percent  of  the  total  TPS  development  cost  (see  Figure  4-3). 


FIGURE  4-3.  TPS  DEVELOPMENT  COST  DISTRIBUTION 


NOTE:  Empirical  data  based  on  NAVAIR  TPS  projects  that  naturally  fall  into  two  distinct 
groups. 

SOURCE  F  Liguori,  "Understanding  and  Controlling  ATE  Software  Costs,”  Automatic  Test¬ 
ing  '76  Conference  Proceedings.  March  1976 

Although  TPS  integration  requires  expert  skills,  many  of  the  major  problems  in  practice 
are  related  to  the  availability  of  ATE  machine  time,  UUTs,  and  associated  repair  parts  (to  repair  a 
UUT  after  fault  insertion).  The  ATE  machine  time  required  for  TPS  verification  is  considerable. 
One  rule  of  thumb  is  l  hour  for  each  test  (defined  as  comprising  all  test  program  statements 
associated  with  setting  up  stimuli,  routing  the  signals,  executing,  and  evaluating  one  measurement 


of  a  UUT  parameter).  Since  many  TPSs  contain  hundreds  of  those  tests  for  a  medium-compiexity 
UUT,  ATE  availability  can  thus  easily  become  a  bottleneck  in  TPS  development.24 

A  more  fundamental  problem  is  that  many  of  the  TPS  acceptance  test  procedures  in 
current  use  have  not  been  validated,  thus  creating  an  unwarranted  confidence  in  the  quality  of  TPSs 
passing  the  test.  This  shortcoming  may  explain  the  lack  of  aggressive  action  by  the  Military  Services 
in  monitoring  performance  of  fielded  TPSs.  To  illustrate,  the  acceptance  test  procedure  specified  in 
the  Navy’s  MIL-STD-2077  is  based  on  a  statistical  sampling  plan  with  the  following  parameters: 

•  Acceptance  quality  level  (defined  as  "the  maximum  percent  defective  that  for  pur¬ 
poses  of  sampling  can  be  considered  satisfactory  as  a  process  average”)  of  2. 5  percent. 

•  Limiting  quality  (defined  as  "the  worst  product  quality  that  the  customer  is  willing 
to  accept”)  of  13  percent. 

•  Consumer’s  risk  (probability  that  a  bad  TPS  is  accepted)  equals  the  producer’s  risk 
(probability  that  a  good  TPS  is  rejected),  with  the  percentage  varying  from  1  percent 
to  25  percent  as  a  function  of  the  UUT’s  reliability 

While  this  sampling  plan  may  be  practical  for  hardware,  it  is  inappropriate  for  software, 
where  the  number  of  possible  test  program  deficiencies  is  more  a  function  of  UUT  complexity,  test¬ 
ability,  and  compatibility  with  ATE,  than  of  UUT  reliability.  (Although  the  statistical  laws  of 
quality  control  stipulate  that  the  level  of  protection  or  risk  afforded  by  a  sampling  plan  depends  on 
the  absolute,  not  relative,  size  of  the  sample,  the  notion  that  a  sample  should  be  a  certain  percentage 
of  the  population  is  a  common  misperception.)  The  logic  of  MIL-STD-2077  leads  both  to  an  excessive 
consumer’s  risk  and  a  poor  quality  standard,  as  the  following  example  illustrates 

For  a  UUT  with  a  mean  time  between  maintenance  actions  of  176  to  200  hours,  MIL- 
STD-2077  (Appendix  F)  specifies  a  consumer’s  and  producer's  risk  of  15  percent.  The  resulting 
sequential  sampling  chart  is  shown  in  Figure  4-4.  The  minimum  number  of  faults  to  be  inserted  for 
acceptance  testing  is  15,  given  the  sampling  plan  parameters  stated  in  Figure  4-4  and  the  formulas 

24Empirical  data  show  that  average  ATE  time  for  test  program  validation  varies  between  0  6 
and  1.2  hours  per  test.  For  large-scale  digital  test  programs  (with  thousands  of  tests),  less  ATE  time 
is  required  due  to  off-line  program  verification  via  simulation.  Source:  F  Liguori,  "Understanding 
and  Controlling  ATE  Software  Costs.”  Automatic  Testing  '76  Conference  Proceedings.  March  1976 


given  in  MIL-STD-2077.  If  the  TPS  detects  and  isolates  each  of  these  first  15  faults  successfully 
(within  agreed  ambiguity  levels),  it  passes  the  acceptance  test.  If  it  misses  one  or  two  faults,  testing 
continues  until  a  maximum  number  of  23  fault  insertions.  If  by  that  time  the  TPS  has  missed  no 
more  than  two  faults,  it  passes  the  test;  otherwise,  it  is  rejected. 


FIGURE  4-4  SEQUENTIAL  SAMP  JNG  CHART  FOR  TPS  ACCEPTANCE  TEST 


V 


(Per  MIL-STD-2077) 
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While  this  example  shows  some  of  the  weaknesses  in  the  TPS  acceptance  test,  the  weak¬ 
nesses  are  actually  understated  because  the  inserted  failures  are  selected  by  the  procuring  activity 
from  a  contractor-prepared  fault  sample  selection  list,  even  though  the  military  standard  points  out 
that  "the  Government  reserves  the  right  to  select  faults  not  appearing  on  the  fault  list  provided  by  the 


contractor").  Furthermore,  one  of  the  TPS  quality  metrics,  i.e.,  run  time,  is  never  considered  in  the 
acceptance  test. 

The  preceding  material  shows  that  many  of  the  current  practices  for  testing  TPSs  have 
serious  weaknesses  and  that  monitoring  the  performance  of  fielded  TPSs  is  crucial  for  quality  control. 
(We  do  not  believe  that  additional  testing  will  improve  TPS  performance;  high-quality  TPS  perfor¬ 
mance  can  only  occur  through  improved  design,  not  by  increased  testing.)  To  illustrate  the 
importance  of  monitoring  TPS  performance,  every  TPS  that  passes  the  acceptance  test  described  in 
the  above  example  may  not  detect  13  percent  of  the  faults  in  the  UUT,  with  a  15-percent  probability 
that  it  may  perform  worse;  its  Type  I  error  rate  (declaring  good  UUTs  faulty)  is  unknown  because  it 
was  not  evaluated;  its  fault  isolation  resolution  for  the  thousands  of  possible  faults  not  evaluated  in 
the  acceptance  test  may  be  much  worse  than  expected;  and  its  run  time  may  be  much  longer  than 
necessary.  Moreover,  certain  failure  modes  for  digital  UUTs,  such  as  dynamic  (timing)  faults,  cannot 
be  inserted  or  are  very  difficult  to  insert;  those  modes  represent,  however,  a  significant  portion  of  the 
digital  module  faults  encountered  in  the  operational  environment. 

TPS  OPERATION  AND  SUPPORT 

The  specific  procedures  for  fielding  and  supporting  TPSs  differ  among  and  within  the  Military 
Services,  but  the  generic  process  is  the  same  Each  first  production  TPS  that  passes  the  acceptance 
test  receives  a  physical  configuration  audit,  after  which  operational  TPSs  are  produced  and  distrib¬ 
uted  to  field  and/or  depot  maintenance  activities  Audits  are  performed  in  accordance  with  standard 
procedures.25  At  the  time  of  fielding  (Army,  Navy)  or  sometime  thereafter  (Air  Force),  management 
responsibility  is  transferred  from  the  developer  to  the  cognizant  support  activity.  The  latter  is 
normally  the  same  activity  that  is  responsible  for  life  cycle  management  of  the  supported  UUT  The 
Army  recently  centralized  TPS  management  oversight,  including  establishment  of  policies  and 
procedures,  under  the  central  TPS  product  manager  within  the  PM,  TMDE  Office  The  Navy  has  a 

25See,  for  example,  M1L-STD-1521 A  (USAF),  "Technical  Reviews  and  Audits  for  Systems. 

Equipments,  and  Computer  Programs,”  1  June  1976  (Notice  2,  21  December  1981) 


similar  structure  but  at  the  Systems  Command  level  (with  NAVELEX  the  lead  Systems  Command 
for  policy  and  testing  technology).  In  contrast,  the  Air  Force  has  dispersed  that  oversight 
responsibility  among  the  ALCs  (with  policies  and  procedures  coordinated  through  AFLC  Head¬ 
quarters  and  Air  Force  Acquisition  Logistics  Center). 

TPS  support  includes  software  maintenance  (correcting  errors  or  voids  in  the  test  program, 
improving  the  performance  of  the  test  program,  or  revising  the  test  program  as  a  result  of  UL’T  con¬ 
figuration  changes  or  engineering  changes  or  because  of  ATE  changes),  technical  documentation 
maintenance,  and  hardware  maintenance  (repairing  the  interface  device,  fielding  engineering 
changes  to  improve  its  reliability  or  maintainability,  or  developing  changes  required  to  accommodate 
ATE  changes  or  test  program  revisions).  One  practical  distinction  between  CUT  and  TPS  support  is 
that  UUTs,  once  they  have  been  fielded,  normally  receive  at  most  a  few  engineering  changes  during 
their  life  cycle,  whereas  TPS  changes  are  a  continuing  requirement  throughout  their  life  cycle,  posing 
significant  management  challenges. 

Past  experience  with  field-level  ATE  has  revealed  serious  problems  in  the  operation  and  sup¬ 
port  of  ATE,  inhibiting  achievement  of  acceptable  or  planned  levels  of  productivity  Many  of  those 
problems,  however,  cannot  be  attributed  solely  to  TPS  deficiencies  A  1980  survey  of  ATE  sites  rated 
ATE  software  (i.e  ,  system  software  and  test  programs!  near  the  bottom  (sixth  out  of  seven)  of  problem 
categories.26  According  to  that  survey,  the  five  higher  ranked  problem  categories  were  as  follows  in 
order  of  importance:  personnel  (inadequate  skill  level,  training,  and  motivation);  operating  proce¬ 
dures  ( lack  of  automated  management  information  systems);  supply  ( insufficient  spares  for  UUT  and 
ATE  repairs);  ATE  hardware  (proliferation  of  large,  single-ported  stations,  too  many  and  too  complex 
interface  devices,  operationally  unsuitable  equipment,  and  inadequate  calibration  procedures),  and 
facilities  (cramped  space  and  inadequate  power  and  air  conditioning)  The  seventh  problem  category. 


26The  survey  was  conducted  in  the  context  of  the  Industry. '.Joint  Services  Automatic  Test 
Project  that  is  described  in  Volume  II  of  this  report  A  summary  of  the  survey  findings  is  provided 
in  Victor  A  Bloom.  "The  LMtimate  ATE  Customer  -  The  L'ser."  AL  TOTESTCON  '81 
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the  only  one  rated  as  less  serious  than  TPS  deficiencies,  was  publications  (manuals  arriving  too  late, 
out  of  date,  not  matching  the  configuration  of  the  UUT,  or  poorly  organized) 

Although  the  management  of  TPS  support  may  not  be  the  weakest  link  in  the  chain  of 
resources  associated  with  ATE  operation  and  support,  we  believe  there  is  ample  opportunity  as  well 
as  need  for  improvements.  Four  management  aspects  that  are  critical  to  TPS  support  are  logistic 
support  planning  and  implementation,  deficiency  reporting  and  correction,  configuration  manage 
ment,  and  TPS  performance  monitoring.  Those  four  aspects,  as  well  as  the  problems  in  TPS 
acquisition  outlined  above,  need  to  be  accorded  increased  management  emphasis  to  effect  the  poten¬ 
tial  improvements  in  productivity  proffered  by  automatic  testing  and  to  realize  a  return  that  is  pro¬ 
portionate  to  the  high  investment  in  ATE  and  TPSs. 

Adequacy  of  1LS  Plan  and  Implementation 


Of  all  the  TPS  standards,  specifications,  manuals,  and  guides  that  are  in  current  use, 
only  one  mentions  the  need  for  an  ILS  package  for  the  TPS  and  describes  the  contracting  for,  and 
review  of,  such  a  support  package  to  ensure  organic  supportability  of  the  TPS  That  document  was 
prepared  by  the  Navy’s  Test  Technology  Information  Center  (an  activity  chartered  by  NAVELEX  to 
acquire,  organize,  store,  and  disseminate  research,  development,  test,  and  evaluation  information  in 
the  testing  area)  as  a  guide  for  NAVSEA  27  Not  surprisingly,  without  such  guidance,  some  ILS 
elements  are  often  ignored  in  the  TPS  acquisition  process.  They  include  maintenance  and  supply 
support  for  interface  devices,  support  equipment  needed  to  operate  the  TPS  in  a  calibration  mode, 
telecommunications  facilities  for  test  program  distribution  or  modification,  and  personnel  and  train¬ 
ing  Moreover,  as  the  Navy’s  guide  suggests,  TPS  support  funds  need  to  be  separately  identified  in 
the  overall  prime  system  budget,  as  well  as  in  the  ATE's  ILS  plan,  and  TPS  acquisition  plan,  to  assure 
that  needed  funds  are  protected.  Lack  of  funding  is  frequently  the  excuse  for  inability  to  support 
TPSs  when  thev  are  fielded 


27Test  Technology  Information  Center,  Test  Program  Sets— Life  Cvcle  Costs.  Technical 
Report  TM-824- 1636  (Corona,  California:  Fleet  Analysis  Center,  15  March  1982) 


i  *  <%  *  •  *  ■  *  »N  !  »  ,  '  »'•  i,'**  *»  iN  k  \  *»  *•  , 


4-30 


Thoroughness  of  Deficiency  Reporting  and  Correction 


Regardless  of  how  well  a  test  program  is  checked  out  during  integration  and  testing, 
problems  will  be  experienced  when  the  TPS  is  distributed  to  maintenance  activities  and  put  into 
operational  use.  The  most  common  problems  include  UUT  failure  modes  that  are  not  or  incorrectly 
isolated,  false  alarms  that  result  from  overly  tight  tolerances  or  unanticipated  noises  and  transients, 
and  operator  errors  that  are  caused  by  inadequate  instructions,  misinterpretation,  or  lack  of  training. 
The  feedback  of  information  obtained  from  analyzing  such  TPS  deficiencies  is  the  principal  mecha¬ 
nism  for  improving  a  test  program. 

Each  of  the  Military  Services  has  instituted  some  sort  of  quality  deficiency  reporting  to 
capture  and  correct  TPS  deficiencies.  The  level  of  detail  and  thoroughness,  however,  varies  consider¬ 
ably.  In  many  cases,  the  system  does  not  go  beyond  the  standard  materiel  deficiency  reporting  proce¬ 


dure;  i.e.,  military  operators  record  materiel  deficiencies  and  forward  that  data  to  central  collection 
points  on  a  routine  basis  for  further  analysis  and  corrective  action.  In  others,  specifically  assigned 
personnel  record  all  failure  events  in  a  detailed  way  and  provide  follow-up  through  systematic  fault 
analysis  and  error  or  deficiency  correction.  This  approach  often  makes  use  of  contractor  personnel, 
while  the  reporting  and  correction  tasks  may  be  incorporated  in  the  TPS  development  contract  under 
appropriate  warranty  provisions  for  a  predetermined  warranty  period,  normally  12  months. 

The  primary  problem  with  using  standard  reporting  procedures  is  that  they  are  not  very 
effective  in  the  case  of  TPSs.  In  contrast  to  prime  equipment  hardware  problems,  TPS  deficiencies  are 
often  difficult  to  pinpoint.  Many  user  sites  simply  lack  the  expertise  to  determine  whether  TPS 
problems  are  due  to  test  program  deficiencies  or  to  a  host  of  other  potential  causes  ( ATE  hardware  or 
software,  UUT  design,  interface  device,  inadequate  operating  procedures,  environmental  factors, 
etc  ).  As  a  result,  users  are  either  reluctant  to  report  observed  problems  with  TPSs  or  they  are  unable 
to  report  anything  beyond  the  fact  they  are  encountering  problems  In  either  situation.  TPS  man 
agers  do  not  receive  the  feedback  needed  to  take  effective  corrective  action  The  customary  practice  of 
the  Military  Services,  therefore,  has  been  to  rely  on  "tiger  teams,"  dispatched  to  field  sites  to  resolve 
ATE/TPS- related  problems. 
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The  key  document  for  configuration  management  of  all  DoD  materiel,  including  non- 
data-processing  computer  programs,  is  DoD-STD-480A,  "Configuration  Control  —  Engineering 
Changes,  Deviations  and  Waivers,”  12  April  1978  (Notice  1,  29  December  1978).  It  defines  config¬ 
uration  management  in  accordance  with  DoD  Directive  5010.19,  "Configuration  Management”  as 

A  discipline  applying  technical  and  administrative  direction  and 
surveillance  to  (a)  identify  and  document  the  functional  and  physical  char¬ 
acteristics  of  a  configuration  item,  (b)  control  changes  to  those  character¬ 
istics,  and  (c)  record  and  report  change  processing  and  implementation 
status. 

As  it  relates  to  automatic  testing,  configuration  management  involves  the  hardware  and  software  of 
the  ATE,  CUT,  and  TPS.  Effective  configuration  management  of  each  item  is  critical  to  the  overall 
maintenance  process  as  well  as  the  performance  of  TPSs. 

Configuration  management  under  the  predecessor  standard  was  flawed  because  it 
ignored  the  impact  of  UUT  engineering  changes  on  test  programs  For  example,  engineering  changes 
not  affecting  form/fit/function  were  always  classified  as  Class  11  engineering  changes  that  were  not 
subject  to  detailed  review  and  documentation  efforts.  As  a  result,  such  configuration  changes  were 
often  only  discovered  after  the  first  failure  in  the  field  when  the  TPS  did  not  work.  The  1978  revision 
of  the  standard,  DoD-STD-480A,  was  designed  to  correct  that  deficiency  It  implies  that  engineering 
changes  shall  be  classified  as  Class  I  when  maintenance  of  computer  programs  or  compatibility  with 
support  equipment  will  be  affected. 

Although  the  appropriate  guidance  on  configuration  management  is  in  place,  the 
Military  Services  continue  to  have  serious  problems  with  configuration  management  of  TPSs.  For 
example,  the  Air  Force  found  that  40  percent  of  the  TPSs  do  not  function  when  they  are  fielded  at  the 
ALCs.28  Similar  Navv-wide  data  are  not  available,  but  the  Trident  Repair  Facility  reported  in 
October  1982  that  97  out  of  106  TPSs  for  the  AN/UYK-7  computer  could  not  test  the  circuit  boards  on 

28Air  Force  Logistics  Command,  A  Study  of  Embedded  Computer  Systems  Support.  9  vols 
(Wright-Patterson  Air  Force  Base,  Ohio:  Air  Force  Logistics  Command,  September  1980) 


hand,  and  the  L'SS  ENTERPRISE  reported  in  February  1983  that  50  percent  of  the  TPSs  for  the 
AN/UYK-20  computer  could  not  test  the  circuit  boards  on  hand.  In  the  Army,  the  TPSs  developed  for 
the  Firefinder  radars  (AN/TPQ-36  and  -37)  had  to  be  redone  after  the  equipment  was  fielded  because 
they  did  not  work.  In  all  of  these  cases,  the  problem  was  largely  attributed  to  inadequate 
configuration  management. 

Although  the  Military  Services  have  tightened  their  configuration  management  policies 
and  procedures,  one  significant  problem  remains  — the  time  delay  involved  in  formal  configuration 
management  procedures  combined  with  the  frequency  or  volume  of  test  program  software  changes. 
From  the  time  a  TPS  deficiency  is  observed  and  reported,  it  may  take  over  a  year  to  fix  it  and  field  the 
improved  version.  Consequently,  the  Military  Services  have  tried  to  evolve  configuration  manage¬ 
ment  procedures  that  are  better  tailored  to  software  maintenance  The  standard,  as  revised, 
recognizes  this  need  by  offering  the  following  guidance  for  computer  program  configuration  items. 

The  configuration  control  over  computer  program  configuration 
items  may  introduce  a  series  of  documentation  terms  and  change  control 
classification  factors  which  differ  from  those  in  DoD-STD-480A  In  such 
cases,  the  standard  should  be  tailored  to  clearly  define  which  documents 
comprise  which  base  line  and  which  factors  control  the  classification  of 
changes  [DoD  STD-480A,  Appendix  F,  p.  731. 

In  August  1980,  the  Air  Force  convened  a  conference  to  address  problems  associated 
with  configuration  management  of  embedded  computer  software.  One  of  the  conclusions  coming  out 
of  that  conference  was  the  following: 

The  DoD  and  subsequently  the  USAF  and  its  major  commands  have 
generated  a  large  number  of  regulations,  directives,  and  standards  that 
define  and  direct  policies  and  procedures  for  accomplishing  configuration 
management  The  majority  of  these  policies  and  procedures  for  accom¬ 
plishing  configuration  management  The  majority  of  these  policies  and 
procedures  are  hardware  oriented  and  have  not  been  tailored  for  software 
Software  considerations  have  been  added  after  the  fact  The  result  is 
that  they  conflict  with  each  other,  use  non-standard  terminology,  and  more 
importantly  do  not  describe  a  consistent  overall  policy  for  configuration 
management  of  embedded  computer  software  and  its  relationship  to  the 
hardware  elements  of  a  particular  system  This  lack  of  consistent  policies 
has  resulted  in  fragmented,  unworkable  configuration  management 
procedures  Several  undesirable  situations  have  resulted.  .  One 
situation  that  consistently  occurs  in  the  area  of  ATE  embedded  computer 
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software  is  the  inability  of  managers  to  provide  corrections  and  updates  to 
the  software  user  in  a  timely,  efficient  manner.29 


This  Air  Force  conference  resulted  in  identifying  a  better  process  for  software  configuration  manage¬ 
ment.  The  recommended  approach,  however,  has  not  yet  been  included  in  the  pertinent  Air  Force 
standard,  MIL-STD-4831USAF),  "Configuration  Management  Practices  for  Systems,  Equipment, 
Munitions,  and  Computer  Programs,”  31  December  1970. 

The  Army’s  approach  to  TPS  configuration  management  is  based  on  a  configuration 
management  plan  established  in  accordance  with  MIL-STD-I456  "Contractor  Configuration  Manage¬ 
ment  Plans.”  It  also  performs  configuration  status  accounting  in  accordance  with  MIL-STD-482, 
"Configuration  Status  Accounting  Data  Elements  and  Related  Features.”  It  exercises  configuration 
control  for  test  program  files  following  the  acceptance  test,  for  TPS  documentation  following  the 
Functional  Configuration  Audit,  and  for  test  accessories  drawings  after  the  Physical  Configuration 
Audit.  Engineering  changes  are  processed  in  accordance  with  DoD-STD-480A. 

The  Navy’s  approach  to  TPS  configuration  management,  as  detailed  in  MIL-STD- 
2077(AS),  "Test  Program  Sets,  General  Requirements  for,”  is  also  based  on  formal  engineering 
change  control  in  accordance  with  DoD-STD-480A. 

Both  Army  and  Navy  are  faced  with  software  configuration  management  problems 
similar  to  those  of  the  Air  Force  because  their  procedures  are  based  on  the  same  hardware-oriented 
standards.  In  response  to  those  problems,  the  Navy  released  MIL-STD-2077A  in  late  1985  as  a  Navy, 
not  just  NAVAIR,  standard.  That  standard  expands  the  TPS  requirements  in  the  areas  of  engi¬ 
neering  support  data,  documentation,  and  identification  (numbering  system)  Similarly,  the  Army 
released  its  new  Army  Test  Policy  and  Procedures  Manual  in  draft  form  in  December  1985  In  spite  of 
these  efforts,  an  effective  test  program  configuration  management  procedure  that  enables  the  fielding 
of  software  updates  in  a  timely  and  efficient  manner,  including  the  "fencing”  of  TPS  maintenance 
funds,  does  not  exist,  and  may  require  development  of  a  separate  DoD  standard 

29The  26-28  August  1980  Air  Force  Acquisition  Logistics  Division  (AFALD)  and  Air  Force 
Test  and  Evaluation  Center  (AFTEC)  Embedded  Computer  System  Software  Configuration 
Management  Final  Report.  21  November  1980 


Monitoring  TPS  Performance 

The  extent  to  which  the  Military  Services  monitor  the  performance  (fault  coverage,  fault 
resolution,  and  run  time)  of  fielded  TPSs  varies  substantially.  For  some  weapons  systems,  a  closed- 
loop  feedback  system  has  been  installed,  as  part  of  the  initial  TPS  fielding  effort,  to  monitor  ATE/TPS 
performance  and  implement  improvements  as  needed.  Those  are  the  exception,  however.  Normally, 
those  feedback  systems  are  discontinued  within  a  few  years  after  fielding.  The  primary  reason  that 
they  are  discontinued  is  cost.  For  example,  the  installation  and  3-year  operating  costs  of  such  a 
system  for  the  F-16  was,  reportedly,  $60  million. 

Apart  from  such  short-term  efforts  for  selected  cases,  none  of  the  Military  Services 
monitors  TPS  performance  on  a  permanent  basis  beyond  the  data  collected  through  their  standard 
maintenance  data  systems.  Those  data,  however,  consist  of  worksheets  filled  out  by  maintenance 
personnel  and  are  subject  to  many  limitations.  The  Air  Force  decided  a  few  years  ago  that  those  data 
would  be  inadequate  for  evaluating  TPS  and  ATE  performance.  AFLC  then  developed  a  management 
information  system  for  ATE,  which  was  implemented  on  a  pilot  basis  in  1982.  The  system  requires 
additional  data  from  the  field  to  augment  that  collected  by  the  standard  maintenance  data  system. 
This  system,  however,  has  not  yet  been  implemented  Air  Force-wide. 

In  contrast  to  the  Air  Force,  neither  the  Army  nor  the  Navy  has  any  plans  to  install  a 
separate  system  to  collect  TPS  performance  data.  While  NAVAIR  has  installed  a  major  management 
information  system  for  retrieving  TPS  field  performance  data  in  various  formats,  the  data  being 
submitted  are  not  of  high  quality  30 

The  lack  of  interest  by  the  Military  Services  in  more  closely  monitoring  TPS  perfor 
mance  may  be  closely  related  to  the  absence  of  analytical  tools  needed  to  assess  TPS  quality  For 
example,  when  they  do  not  know  what  the  fault  coverage,  fault  resolution,  and  run  time  of  a  TPS 
should  be,  the  collection  of  actual  TPS  performance  data  is  of  little  use.  In  the  absence  of  such 
knowledge,  the  data  by  themselves  are  of  little  value  and  certainly  not  worth  the  effort  of  installing  a 

30For  a  description  of  this  system,  see  Ricardo  Springs,  et  al.,  "Automatic  Test  Equipment 
Workload  Model  for  Navy  Intermediate  Maintenance,”  AUTOTESTCON  ’85.  pp  30-36 
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separate  data  collection  system.  Yet,  past  experience  with  TPS  improvement  projects  has  demon¬ 
strated  that  the  performance  of  fielded  TPSs  (especially  run  time)  can  be  improved  substantially 
through  relatively  straightforward  changes  in  the  test  program.  Those  changes  may  be  as  simple  as 
modifying  the  sequence  of  tests  in  accordance  with  failure  frequency.  Two  examples  that  illustrate 
the  potential  increase  in  TPS  are  (1)  the  Navy’s  update  of  TPSs  for  the  S-3A  in  the  late  1970’s  and 
(2)  the  Air  Force’s  update  of  TPSs  for  the  F-15  electronic  warfare  subsystem  in  the  early  1980’s.  In 
both  cases,  TPS  run  times  were  reduced  between  30  to  50  percent  at  relatively  minor  cost.  Even  more 
drastic  improvements,  as  high  as  80  percent,  are  attainable  by  using  any  of  the  test  analysis  tools 
that  have  become  available  in  recent  years.31 


The  message  is  quite  clear.  The  Military  Services  need  to  establish  a  "should-cost/ 
performance”  capability  for  TPSs,  institute  feedback  systems  to  accurately  monitor  TPS  performance 
in  the  field,  and  provide  the  funds  required  to  improve  TPS  performance.  The  feedback  system  does 
not  have  to  rely  on  manual  reporting  by  maintenance  personnel;  it  could  be  a  fully  automated  process 
using  the  ATE  computer  for  accumulating  a  daily  log  of  ATE  usage,  including  identification  and  test 
results  for  each  TPS  execution,  with  the  daily  logs  from  all  ATE  sites  transmitted  to  a  central 
processing  activity.  These  actions  have  the  potential  to  resolve  many  of  the  current  problems  of 


inadequate  TPS  performance,  thereby  improving  the  return  on  the  DoD’s  investment  in  TPSs. 
SUMMARY 

This  chapter  has  focused  on  only  a  few  aspects  of  TPS  acquisition  and  support  management  and 
the  associated  policies,  procedures,  and  practices.  Notwithstanding  this  coverage,  it  is  clear  that 


significant  management  improvements  are  called  for. 

TPS  acquisition  is  as  much  a  black  art  as  it  was  10  or  20  years  ago.  This  is  best  illustrated  by 
the  TPS  acquisition  history  for  the  F/A-18  program.  When  the  prime  contractor  quoted  a  bid  well 


beyond  budget,  the  Navy  competed  the  development  of  most  TPSs.  The  winning  bids  (for  each  of 
three  groups  of  TPSs  competed  separately)  resulted  in  a  contract  cost  savings  of  50  percent 


31See  Dr  K  R  Pattipati,  et  al  ,  "Time-Efficient  Sequences  of  Tests  (TEST),”  AL'TOTEST 
CON  ’85.  dp.  49-62 
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(approximately  $125  million).  While  the  Navy  proudly,  and  justifiably  so,  points  to  this  case  as  an 
illustration  of  the  benefits  from  competition  in  TPS  development,  we  believe  it  demonstrates,  as  well, 
the  flaws  in  TPS  contract  specifications:  Anytime  the  bids  submitted  by  technically  well-qualified, 
reputable  firms  vary  by  such  a  large  amount,  one  conclusion  to  be  drawn  is  that  the  statement  of  work 
is  too  ambiguous.  One  of  the  primary  causes  for  such  ambiguity  is  that  the  Military  Services  have  not 
collected  the  necessary  data  on  TPS  acquisitions  and  field  performance;  they  also  have  not  developed 
the  analytical  tools  needed  to  relate  TPS  cost  and  performance  to  UUT  and  ATE  characteristics 

There  is  no  question  that  the  Military  Services  are  encountering  serious  problems  with  TPS 
performance.  Many  TPSs  simply  do  not  work  when  they  are  fielded  When  and  if  they  are  made  to 
work,  fault  coverage  and  fault  resolution  are  frequently  well  below  expectations,  and  run  time  is 
mostly  ignored.  The  primary  reason  for  low  TPS  quality  can  be  traced  to  inadequate  procedures  and 
tools.  Passing  the  TPS  acceptance  test  is  a  necessary  condition  for  quality  certification,  but  it  is 
certainly  not  a  sufficient  condition.  The  phased  review  process  instituted  by  the  Navy  for  TPS 
development  is  surely  a  step  in  the  right  direction,  but  it  cannot  be  fully  effective  because  the  Navy 
cannot  verify  that  the  starting  document,  the  TRD,  is  complete  and  accurate.  Similarly,  the  applica¬ 
tion  of  TPS  development  tools  has  focused  largely  on  ATPGs,  but  there  are  other  test  analysis  and 
simulation  tools  that  need  to  be  exploited  to  optimize  TPS  performance.  TPS  development  contracts 
should  not  merely  include  performance  goals  but  also  hard  requirements,  with  a  test  improvement 
warranty  concept  similar  to  that  used  for  hardware  reliability  improvements.  Once  fielded,  TPS 
quality  must  be  closely  monitored  through  a  closed-loop  feedback  system  and  improved  in  all  three 
respects  (fault  coverage,  isolation,  and  run  time)  until  it  meets  contractual/planned  performance 
levels.  Such  monitoring  needs  to  be  completely  automated,  using  the  ATE  computer,  instead  of 
relying  on  manual  job  sheets.  Only  after  a  TPS  has  been  operationally  certified  as  meeting  requisite 
performance  levels,  should  life  cycle  management  be  turned  over  to  the  user 

TPS  support  has  improved  recently  because  of  the  lessons  learned  in  the  1970’s  when  the 
Military  Services  tried  to  cut  costs  by  not  procuring  the  technical  data  (TRDs  and  supplementary 
L’UT  technical  data)  needed  for  TPS  maintenance  However,  some  of  the  support  problems  caused  by 


ignoring  other  ILS  elements  have  received  less  attention.  Additionally,  current  configuration  man¬ 
agement  procedures  are  not  well  suited  to  cope  with  the  volume  and  frequency  of  software  changes, 
nor  are  current  budgeting  procedures  adequate  to  provide  for  ongoing  software  maintenance  funds 
Test  program  maintenance  tools,  such  as  engineering  work  stations  with  associated  data  bases,  would 
facilitate  TPS  maintenance  and  increase  productivity  of  scarce  personnel,  but  the  Military  Services  so 
far  have  not  invested  in  those  tools.  In  short,  TPS  support  continues  to  rely  on  crisis  management, 
not  on  a  well-planned  life  cycle  program. 

The  above  assessment  of  the  Military  Services’  management  of  the  acquisition,  quality  assur¬ 
ance,  and  support  of  TPSs  is  shared  by  a  number  of  industry  representatives.  They  believe  that  their 
consensus  is  that  current  DoD-wide  problems  with  TPSs  are  serious  and,  unless  changes  are  made, 
the  problems  will  become  worse  with  the  next  generation  of  high-technology  weapons  systems  Some 
of  the  Military  Services  question  the  affordability  of  correcting  current  problems  As  a  result,  they 
have  shifted  their  emphasis  toward  development  of  a  new  approach:  integrated  diagnostics  The 
integrated  diagnostics  concepts  may  well  be  the  solution,  but  does  not  extend  to  currently  fielded 
weapons  systems  that  may  remain  in  the  DoD  inventory  through  the  late  1990’s. 
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APPENDIX  A 


ARMY  PROGRAM  MANAGER  CHARTER  FOR  TEST  EQUIPMENT 

This  appendix  contains  the  current  charter  of  the  Army’s  program  manager  for  test,  measure¬ 
ment,  and  diagnostic  equipment  (PM,  TMDE).  The  charter  was  first  established  in  April  1982, 
following  Secretary  of  the  Army  approval  of  the  recommendations  of  the  DATAT  (Department  of  the 
Army  TMDE  Action  Team)  Report  and  the  resulting  establishment  of  a  centralized  TMDE  manage¬ 
ment  structure  per  Secretary  of  the  Army  charter  for  the  "U  S  Army  Executive  Agent  for  TMDE,” 
27  April  1982. 

The  current  PM,  TMDE  charter  differs  slightly  from  an  earlier  one  in  that  the  PM’s  "reporting 
channel”  (Section  I)  has  been  upgraded  to  Executive  Director  for  TMDE,  and  his  "direct  communi¬ 
cation  channel”  (Section  VI  B)  to  Chief  of  Staff,  Army  Furthermore,  field  offices  created  by  the  PM 
now  need  to  be  approved  only  by  the  Executive  Director  for  TMDE  (Section  VII).  Thus,  both  the 
influence  and  leverage  of  the  PM,  TMDE  have  increased. 

The  following  pages  have  been  retyped  from  the  original  copy  received  from  the  Army.  We 
have  excluded  Annex  B,  "Interface  and  Participating  Organizations  ” 


PROGRAM  MANAGER  CHARTER 


TEST,  MEASUREMENT  AND  DIAGNOSTIC  EQUIPMENT 


I  DESIGNATION  OF  PROGRAM  MANAGER 


Colonel  Douglas  H.  Barclay  is  designated  Department  of  the  Army  (DA)  Program  Manager 
(PM)  for  Test,  Measurement  and  Diagnostic  Equipment  (TMDE).  Colonel  Barclay  assumed  project 
responsibility  on  12  April  1984  The  PM  reports  to  the  Executive  Director  for  TMDE  (EDT),  through 
the  Commanding  General  (CG),  U  S.  Army  Communications-Electronics  Command  (CECOM) 

II.  MISSION 


The  PM  is  responsible  for  the  management  of  Army  TMDE  in  accordance  with  Department 
of  Defense  Directive  (DoDD)  5000.1,  5000.37,  5000 .39,  and  5000.40;  Army  Regulations  (AR)  1000-1, 
750-43,  750-1,  702-3,  700-129,  700-127,  70-61,  70-17,  and  11-18;  DARCOM-R  715-2,  700-15,  70-1,  and 
11-16;  and  other  pertinent  regulations  for  program  management  of  the  TMDE  Program  The  TMDE 
Program  includes  the  Product  Manager,  Automatic  Test  Support  Systems  (PM,  ATSS),  who  is 
responsible  for  centralized  life  cycle  management  of  the  Army’s  automatic  test  support  systems  and 
equipments;  the  Product  Manager,  Army  TMDE  Modernization  (PM,  TEMOD),  who  is  responsible  for 
centralized  life  cycle  management  of  the  Army’s  electronic  TMDE  Program,  future  developmental 
and  nondevelopmental  items  of  general-purpose,  electronic  TMDE,  and  expansion  of  the  TMDE 
Modernization  Program  to  all  classes  of  TMDE;  and  the  Army  Test,  Measurement  and  Diagnostic 
(TMD)  Technology  Laboratory  (ATTL),  which  is  responsible  for  the  Army  TMDE  technology  base  and 
the  Army  Standard  ATE  programming  language. 

III.  AUTHORITY  AND  RESPONSIBILITY 

A.  Authority 

The  PM  has  been  delegated  the  full  line  authority  of  the  Department  of  the  Army  TMDE 
Executive  Agent  (DATEA),  as  delegated  to  the  EDT,  for  the  centralized  management  of  the  Army’s 
Automatic  Test  Support  System,  TMDE  Standardization  and  Modernization  Programs,  technology- 
base  programs  and  standard  Army  ATE  language 

B.  Responsibilities 

1  Receive,  interpret  and  implement  overall  TMDE  policy  as  issued  by  the  EDT 
Assure  that  these  policies  are  implemented  within  his  areas  of  responsibility  and  that  each  new 
system,  beginning  with  concept  formulation,  has  a  TMDE  pre-in  process  review  at  each  major  deci¬ 
sion  point  to  insure  integration  of  TMDE  technology 

2.  Compile,  develop  and  submit  to  the  EDT  the  Army  TMDE  Five-Year  Program  Plan 
(FYPP)  in  support  of  the  Program  Objective  Memorandum  (POM)/Budget  Submission.  Insure  the 
program  plan  addresses  the  Research,  Development,  Test,  and  Evaluation  (RDTE)  Program,  the 
Army  Procurement  Appropriations  Program  and  the  Force  Modernization  Program. 

3  Develop,  prepare,  coordinate  and  defend  a  consolidated  program  for  research,  develo- 
ment  and  procurement  of  all  Army  general-purpose  TMDE/ATE  Insure  this  program  is  fully  deve- 
oped  and  provided  to  the  EDT  for  inclusion  in  the  DATEA  total  Army  TMDE  fiscal  submission 
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4.  Develop  and  maintain  the  Army  TMDE  technology  base  and  manage  associated 
programs  Serve  as  the  Army's  central  TMDE  research  and  development  activity 

5  Manage  the  modernization  and  standardization  of  the  Army  general  purpose 
TMDE/ ATE  inventory 

6  Develop  and  ;mplement  policy,  pro  -dures  and  standards  for  development  and  field 
ingofTest  Program  Sets  (TPS's)  for  weapon  systems 

7  Assess  Army  mission  and  weeoon  systems’  test,  measurement,  and  diagnostic  needs 
within  his  area  of  responsibility.  Initiate  £  cion  to  satisfy  identified  needs  and  submit  recommenda¬ 
tions  to  the  EDT  for  those  needs  outside  the  PM’s  areas  of  responsibility 

8  Manage  the  development  of  the  Army  Standard  ATE  language  and  insure  life  cycle 
configuration  control  as  outlined  in  Annex  A. 

9  Prepare  technical  guidance  for  DA  and  DARCOM  PMs  and  commodity  commanders 
in  the  design  and  development  of  TMDE  Maintain  an  engineering  technology  center  of  excellence  for 

TMDE 

10  Oversee  development  of  general-purpose  TMDE/ATE 

11  Coordinate  all  PM  policies  and  procedures  with  the  EDT  and  other  appropriate 

agencies 

12  Support  the  EDT  on  Army,  DoD  and  industrial  panels  relating  to  TMDE 

IV  RESOURCE  CONTROL 

A.  The  PM  will  insure  that  dollar  and  manpower  requirements  to  accomplish  the  above 
responsibilities  are  developed  and  submitted  in  accordance  with  established  DoD/DA/lMRCOM 
manpower/funding  channels  and  procedures  for  inclusion  in  the  Program  Analysis  Resources  Review 
(PARR)/Modernization  Resource  Information  Submission  (MKIS)  for  applicable  target  program 
years;  and  that  RDTE,  procurement,  operation  and  maintenance  and  sUrk  fund  requirements  are 
compatible  at  all  times  with  life  cycle  progression  of  the  programs 

B.  Army  and  other  departmental  monetary  resources  approved  to  accomplish  the  PM 
mission  will  be  provided  to  the  PM  or  participating  organizations  having  prime  mission  or  task 
responsibility  utilizing  established  DoD/DA/DARCOM  funding  channels  and  procedures  The  PM' 
will,  in  turn,  provide  the  necessary  funding,  direction  and  guidance  in  accordance  with  current 
regulations,  policies  and  procedures.  Organizations  receiving  resources  will  be  required  to  furnish 
such  status  or  progress  reports  to  the  PM  as  are  necessary  in  the  execution  of  his  mission 

C  The  PM  is  responsible  foi  reporting  through  appropriate  DARCOM  Comptroller 
channels  those  Foreign  Military  Sales  (FMS)  costs  incurred  by  the  Office  of  the  Program  Manager 
Where  the  PM  tasks  commands,  agencies,  or  activities  to  perform  at  his  request,  such  taskings  will 
include  FMS  cost  reporting  as  appropriate.  Tasked  activities  are  responsible  for  accuracy  and  ade¬ 
quacy  of  their  incurred  costs.  Supply  and  financial  arrangements  between  participating  foreign 
governments  and  the  U  S  Army  will  utilize  FMS  procedures  and  be  documented  through  the 
U  S.  Army  Security  Assistance  Center 


STANDARDIZATI0N/INTER0PERAB1HTY 
The  PM  will: 

A.  Support  the  EDT  on  matters  involving  Standardization  and  Interoperability  (SI)  of 
TMDE,  and  assist  in  assuring  that  SI  actions  are  accomplished  in  a  timely  manner 

B.  Analyze  the  TMDE  requirements  and  developments  proposed  to  support  primary 
systems  used  jointly  by  the  U  S.  and  other  countries.  Identify  possibilities  for  standardizing  such 
TMDE. 

C.  Explore  the  technical  feasibility  of  interoperating  test  software  on  different  auto¬ 
matic  test  equipment  used  by  the  U  S.  and  other  countries. 

D.  Insure  NATO  and  other  countries’  SI  considerations  are  adequately  addressed  and 
presented  at  all  major  program  reviews. 

VI.  COMMUNICATION  CHANNELS 

A.  Direct  communication  is  authorized  between  all  participants  involved  in  imple¬ 
mentation  of  the  program  to  assure  timely  and  effective  direction  and  interchanges  of  information 
among  participants. 

B.  The  PM  has  a  direct  channel  of  communication  to  the  Chief  of  Staff,  Army  and  the 
Secretary  of  the  Army  should  any  of  the  participating  organizations  fail  to  respond  to  project  require¬ 
ments  in  any  of  his  assigned  management  areas. 

VII.  LOCATION  AND  SUPPORT 

The  PM  TMDE  office  is  located  at  HQ,  CECOM,  Fort  Monmouth,  New  Jersey,  with 
necessary  facilities,  administrative  and  technical  support  being  provided  by  that  command  Field 
offices  may  be  created  by  the  PM  as  required  and  approved  by  the  EDT  without  change  of  charter. 
Attendant  personnel  requirements  will  be  in  accordance  with  applicable  procedures  Facilities  and 
administrative  support  will  be  provided  by  the  command/activity  where  established  in  accordance 
with  separately  negotiated  host/tenant  agreements. 

VIII.  TERMINATION 

A  The  TMDE  Program  is  diversified  and  is  not  easily  measured  for  termination  deci¬ 
sions.  Accordingly,  each  time  this  charter  is  revised  or  updated,  each  reviewing  command,  agency, 
headquarters,  etc.,  will  recommend  to  the  next  higher  authority  their  stated  reasons  why  continued 
intensive  management  techniques  should  or  should  not  be  continued. 

B.  The  PM  currently  estimates  that  consideration  for  termination  decision  should 
begin  on  or  about  FY  2000. 


ANNEX  A 


PROGRAM  MANAGER  CHARTER 


TEST,  MEASUREMENT  AND  DIAGNOSTIC  EQUIPMENT 

A  ASSIGNED  ARMY  RDTE  PROJECTS  AND  TASKS 

The  PM  is  responsible  for  the  following  RDTE  projects  and  tasks; 

Element  Code 

DA  Proiect 

Title 

6  37  48.A 

1X263748.  AJ28 

TMD  Technology  Development 

6.37. 48.A 

1X263748  AJ29 

Automatic  Test  Support  Systems 
(ATSS) 

6.37  48.A 

1X263748.  D244 

ATSS  Language  Utilization  and 
Standardization 

6  47  46  A 

1E464746.D536 

Simplified  Test  Equipment 
Expandable  (STE-X) 

6.47  46A 

1E464746.D537 

Intermediate  Forward  Test 
Equipment  (IFTE) 

B  OTHER  PROCUREMENT.  ARMY  APPROPRIATION 

The  PM  is  responsible  for  overall  procurement  management,  including  product  improve¬ 
ment;  engineering  in  support  of  production,  Producibility,  Engineering  and  Planning  (PEP);  and 
Initial  Production  Facilities  (IPF)  as  required  for  the  following  items  listed  in  the  Army  Materiel 
Plan  (AMP) 


BLIN 

305171 

TMDE  Modernization 

BUN 

405154 

TMDE  Modernization 

BLIN 

205197 

Test  Station,  Electronic  Equip  , 
AN/USM  410 

BLIN 

305240 

Test  Station,  OQ  290(V)1/MSM  105 

BLIN 

305241 

Equip  Repair  Facility,  ERF 

\  7 


OTHER  ASSIGNED  PROGRAMS  AND  TASKS 
The  PM  is  responsible  for: 

1.  Managing  the  overall  integrated  logistic  support  program  for  all  applications  of  his 
assigned  projects  and  tasks. 

2.  Managing  operation  and  maintenance,  stock  fund,  and  military  construction  funds 
as  applicable  and  assigned. 

3.  Reviewing  and  participating,  as  necessary,  in  the  development  and  update  of  the 
AMP  applicable  to  the  TMDE  Program  to  assure  compliance  with  total  Army  objectives. 

4.  Maintaining  full  logistic  support  and  an  adequate  materiel  readiness  evaluation  of 
assigned  systems  by  analysis  of  supply,  maintenance,  and  unit  readiness  reports  and  assuring  initia¬ 
tion  of  corrective  action  on  all  known  or  projected  deficiencies  bv  appropriate  functional  managers 
and  commanders. 

5.  Coordinating  other  customer  and  international  logistic  procurements  including 
coproduction,  in-country  production,  and  the  provision  of  technical  assistance,  as  applicable 
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Other  program  tasks  or  items  as  assigned. 
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APPENDIX  C 


THE  STANDARD  INTEGRATED  SUPPORT  MANAGEMENT  SYSTEM  (SISMS) 

This  appendix  provides  excerpts  from  the  Joint  Service  regulation  on  acquisition  management, 
"Standard  Integrated  Support  Management  System,”  including  Change  4,  17  September  1982.  The 
excerpts  include  the  Table  of  Contents,  Chapter  1,  "Introduction  and  Concept,”  and  Chapter  5, 
"Support  Equipment.” 
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HEADQUARTERS  AIR  FORCE  LOGISTICS  COMMAND  AFLCR/AFSCR  300-24 

Wright -Patterson  Air  Foret  Bate  OH  45433 

HEADQUARTERS  AIR  FORCE  SYSTEMS  COMMAND 
Andrews  Air  Force  Bate  DC  20334 

U  S.  ARMY  MATERIEL  DEVELOPMENT  AND  READINESS  COMMAND  DARCOM-R  700-97 

Alexandria  VA  22333 

NAVAL  MATERIAL  COMMAND  NAVMATIN1ST  4000  38A 

Washington  DC  20368 

HEADQUARTERS.  UNITED  STATES  MARINE  CORPS 
Washington  DC  20380 

Acquisition  Management 

STANDARD  INTEGRATED  SUPPORT  MANAGEMENT  SYSTEM 

This  document  established  the  Standard  Integrated  Support  Management  System  (SISMS)  as  a  management  approach  jointly 
agreed  to  and  implemented  by  DARCOM/NMC/AFLC/AFSC.  It  provides  information  on  the  interrelationships  and  responsibili¬ 
ties  among  the  executive  service,  the  participating  service,  and  the  contractor  for  multi-service  programs:  states  the  agreements 
which  assign  responsibility,  and  desen bes  the  policies  and  procedures  that  affect  organixation  and  preparation  for  implementation 
SISMS:  contains  standardised  contract  items  for  the  logistics  disciplines  and  provides  for  the  use  of  standard  data  item  descrip¬ 
tions.  The  Office  of  Management  and  Budget  iOMB)  approved  the  SISMS  reporting  and  recordkeeping  requirements  upon 
contractors  under  the  authority  of  OMB  No.  0704-0068  expires  31  December  1984. 

NOTE:  DODDs  and  DODIs  (referenced  here)  are  not  available  through  normal  distribution  channels. 
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Chapter  1 

INTRODUCTION  AND  CONCEPT 


1-1.  Scope.  SISMS  must  be  considered  for  application  to  all 
system/ equipment  acquisitions  (paragraph  1-5).  The  policies, 
procedures,  and  contract  requirements  are  implemented  by 
assimilating  or  adapting  them  to  the  logistics  support  planning 
and  management  routine  of  each  of  the  commands.  SISMS  is 
useful  for  individual  system  and  equipment  acquisition  and 
should  be  applied  consistent  with  valid  requirements  of  indi¬ 
vidual  system  and  equipment  acquiaidon  and  should  be  applied 
consistent  with  valid  requirements  of  individual  programs. 
This  approach  makes  it  available  for  use  in  all  acquisitions  in 
consonance  with  chose  Service  requirements  not  covered  by 
SISMS. 


1-2.  Background.  SISMS  was  conceived  to  provide  a  com¬ 
mon  approach  to  planning  and  managing  the  logistics  support 
of  multi-service  systems/ programs.  SISMS  was  developed 
under  the  auspices  of  the  Joint  Logistics  Commanders  (JLC) 
(US  Army  Materiel  Development  and  Readiness  Command 
(DARCOM).  Naval  Material  Command  (NMC).  Air  Force  Lo¬ 


gistics  Command  (AFLC).  and  Air  Force  Systems  Command 
(AFSC))  during  the  1967-69  timeframe,  initially  for  multi¬ 
service  aeronautical  systems  only.  It  later  became  apparent 
that  portions  of  SISMS  could  be  used  on  other  types  of  sys- 
ums/ equipments,  whether  single-  or  multi-service.  In  1970, 
the  JLC  agreed  to  expand  the  applicability  of  SISMS  and 
implement  its  concept  as  a  management  principal.  The  Logis¬ 
tics  SyitemPolicy  Committee  (LSPC).  chaired  bv  6  ASP 
(tlLI.  recogmzea  tne  potential  advantages  of  SISMS  end  Tn 
19 ,1  taibMS  became  nan  nf  ir  v  Inne-renee  plans  for  imnrov- 
ing  the  overall  Department  of  Defense  ( POD)  logistics  system 
(Ubu  Logistic  bystems  Plan  LuGrLAN"  1972-1980,  15 
May  1972).  It  emnhasixed  the  need  for  the  Services  to  appl 
S 
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equipments. 


1-3.  Description/Concept: 

a.  SISMS  is  an  agreement  among  DARCOM,  NMC.  AFLC 
and  AFSC  to  use  s  uniform  approach  to  logistics  planning  and 
management  Joint  Operating  Agreements  (JOA)  form  the 
basis  of  that  agreement  in  those  logistics  disciplines  needed  to 
provide  a  total  logistics  support  package.  The  JO  As  describe 
the  relationships  and  assign  responsibilities  for  planning  and 
managing  logistics  support  for  all  multi-service  system/equip- 
menu  programs.  Contract  ezhibiu.  relaud  military  standards 
(MIL-STD).  and  data  item  descriptions  I  DID)  which  describe 
contractor  responsibilities  in  a  program  are  included  or  refer¬ 
enced  as  appropriate  for  each  of  the  identified  logistics  disci¬ 
plines. 

b  Two  concepu  are  the  foundation  for  the  SISMS  docu- 

menution; 


(I)  Integrated  weapon  support  management  lIWSMt 
Clear  assignment  of  authority  and  responsibility  for  uniform 
planning,  management,  and  acquisition  of  the  logistics  sup 
port  program  for  multi-service  systems  and  equipments. 


(2)  Integrated  logistics  support  (ILS).  Orderly  considera¬ 
tion  and  inurrelationships  of  the  logistics  support  disciplines 
within  the  sysum/equipment  requirements. 

c.  SISMS  builds  on  this  basic  IWSM/ILS  foundation  by 
developing  each  of  iu  logistics  disciplines  for  uniform  applica¬ 
tion  in  a  multi-service  environment.  In  so  doing,  it  stresses 
commonality  of  broad  principles  and  procedures,  but  permiu 
flexible  employment  of  techniques  to  allow  for  individual  ser¬ 
vice  or  program  peculiarities.  SISMS  recognizes  the  authority 
of  the  executive  service  throughout  design,  acquisition,  and 
support,  but  is  sensitive  to  the  requirements  of  the  participat¬ 
ing  service(s). 

d.  A  JLC  Memorandum  of  Agreement  provides  the  overall 
management  principles  for  conducting  multiservice  programs 
(AFSCR/AFLCR  800-2/ AMCR  70-59/NAVMATINST  5000. 
10A).  That  agreement  assigns  responsibilities  to  the  exec¬ 
utive  service,  the  participating  services,  the  program/project 
manager  (P/PM),  and  the  participating  service  semor  rep¬ 
resentative's).  For  program/project-managed  programs. 
SISMS  is  intended  to  complement  this  JLC  agreement.  It  pro¬ 
vides  the  P/PM  with  established  guidelines,  organizational 
relationships,  and  documentation  for  the  logistics  portion  of 
the  program.  SISMS  also  applies  to  acquisitions  that  are  not 
program/project  managed.  In  these  cases,  the  executive  ser¬ 
vice  will  assume  the  responsibilities  assigned  throughout  this 
regulation  to  the  P/PM. 

e.  The  achievement  of  a  common  or  uniform  approach  to  the 
individual  disciplines  of  ILS  permits  the  user  to  select  from 
and  tailor  the  SISMS  to  fit  the  program's  peculiar  require¬ 
ments.  This  flexibility  is  what  makes  SISMS  meaningful  for 
application  m  a  single- service  environment  Its  expansion  to 
permit  application  to  single-service  system/ equipments  elim¬ 
inates  the  need  of  considering  multi-service  programs  as  any¬ 
thing  other  than  slightly  unusual  examples  of  single-service 
systems  Further,  it  minimizes  the  need  for  the  services  to 
maintain  two  separate  approaches  to  ILS  management  with 
their  inherent  impacts  on  organization,  documentation,  and 
management  systems. 

f.  SISMS  consitutes  s  uniform  set  of  policies  and  opera¬ 
tional  methodologies  in  the  ILS  process  for  both  multi-  and 
single- service  system/equipment  acquisitions.  For  those 
single-service  acquisitions  which  latar  become  multi-service 
programs.  SISMS  would  facilitate  the  transition  to  multi- 
service  use.  Where  standardization  of  procedures  isn't  poss¬ 
ible  due  to  peculiar  requirements  based  on  differences  in 
service  roles  and  missions.  SISMS  provides  the  methods  for 
handling  the  interfaces  between  the  Government  and  industry 
and  among  the  DOD  components. 

g.  SISMS  presents  no  new  or  radically  different  logistics 
support  procedures.  The  most  suitable  methods  used  by  one 
or  more  of  the  joint  logistics  commands  have  been  adapted  and 
adopted  in  SISMS  policies  and  procedures. 

1-4.  Objective.  Throughout  the  development,  implementa¬ 
tion.  and  revision  expansion  of  SISMS  the  basic  objectives 
have  remained  consistent  and  valid.  Those  objectives  are  to: 
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a.  Provide  a  single  source  of  information  with  which  the  JLC 
may  control  and  manage  ILS  programs. 

b.  Delineate  the  authority  for.  and  the  responsibilities  and 
relationahipa  between,  the  executive  and  participating  ser¬ 
vices  engaged  in  a  multi-service  program. 

c.  Permit  the  services/commands  to  present  a  “single  face  to 
industry"  through  the  application  of  common  concepts,  pro¬ 
cedures.  and  policies. 

d.  Enhance  support  responsiveness  through  judicious  plan¬ 
ning  and  management. 

e.  Reduce  the  costa  incurred  by  duplication  in  and  among 
the  services  through  the  use  of  common  logistics  procedures. 

f.  Integrate  and  standardize  acquisition  and  logistics  sup¬ 
port  disciplines  inot  properly  time- phased  actions  to  ensure 
weapon  system  and  equipment  readiness. 

1-5.  Policy.  The  SISMS  is  mandatory  on  multiservice 
system/ equipments:  it  is  mandatory  for  consideration  on  all 
other  systems/  equipments  and  must  be  used  where  advan¬ 
tageous.  Its  use  or  nonuse  must  be  documented  according  to 
the  JLC's  internal  procedures. 

1-8.  Implementation/Application: 

a.  Three  principal  interfaces  are  addressed  in  SISMS: 

(1)  Executive/participating  services  on  multi-service 
programs. 

(2)  Executive  service/ contractor  on  both  multi-  and 
single- service  programs. 

(3)  Participating  service/ contractor  on  multi-service  pro¬ 
grams.  SISMS  has  been  designed  to  enhance  the  flow  of  data 
and  information  at  those  interfaces  through  the  adoption  of 
standardized  policy  and  the  application  of  standardized  pro¬ 
cedures.  standard  contract  items,  terms,  conditions,  and  data 
deliverables.  The  contract  requirements.  DfDs,  various  parts 
of  the  SISMS  document,  and  references  are  intended  to  be 
assimilated  into  the  general  and  detailed  contractual  specifi- 
cationa. 

b.  SISMS  does  not  cover  the  complete  spectrum  of  acquisi¬ 
tion  inputs.  There  will  be  occasions  when  the  functional  and 
managerial  elements  of  each  participating  service  will  operate 
under  the  policies,  data,  standards,  etc.  of  the  executive  ser¬ 
vice  according  to  AFSCR/AFLCR  800-2/AMCR  70-59/ 
NAVMATTNST  5000.10. 

c. SISMS  it  intended  to  be  applied  by  the  use  of  its  standar¬ 
dized  contract  requirements  and  related  DIDs  The  contract 
requirements  tell  the  contractor  what  to  do:  the  related  DIDS 
tell  the  contract  what  data  are  to  be  delivered. 

d.  The  principal  documents  incorporated  in  SISMS  by  ref¬ 
erence.  which  individually  address  all  or  part  of  the  acquisition 
logistic  management  process  are  listed  in  attachment  2  The 
list  is  supplemented  by  the  additional  references  of  those  parts 
of  SISMS  which  address  specific  logistics  disciplines. 


1-7.  Summary: 

a.  The  mechanism  through  which  SISMS  operates  the  ILS 
system  is  an  extension  of  the  existing  policies  and  practices  of 
DOD  Directive  4100.35.  Of  primary  importance  in  this  area 
are  SISMS  chapters  2  and  3: 

(1)  Chapter  2.  Integrated  Logistics  Support  Manage¬ 
ment.  identifies  the  organizational  relationahipa  for  managing 
logistics  programs  and  references  the  additional  sources  of 
guidance  and  governing  documentation.  It  addresses  the  sig¬ 
nificant  contents  of  the  integrated  support  plan  iISPV  It  iden¬ 
tifies  the  goals,  policy,  responsibilities,  and  interrelationships 
for  conducting  the  tasks  and  references  the  related  DIDs. 
where  applicable. 

(2)  Chapter  3.  Logistics  Support  Analysis  Policy  and  Guid¬ 
ance.  addresses  the  logistic  support  analysis  (LSAI  (MIL- 
STD-1388)  as  an  analytical  technique  used  by  the  logistics 
manager  to  provide  a  continual  dialogue  between  the  designer 
and  the  logistician.  The  LSA  identifies  the  unpacts  of  support 
alternatives  upon  design  engineering  and  provides  the  primary 
vehicles  for  interfacing  the  related  disciplines  of  SISMS. 

b.  Chapters  4  through  9  discuss  the  logistics  disciplines 
important  in  the  support  of  systems/equipments.  They  include 
the  provisioning  policies  and  procedures  for  spares  and  repair 
parts:  the  equipment,  both  support  and  government-furnish¬ 
ed:  and  inventory  management  procedures.  They  also  include 
procedures  for  planning  for  the  facilities  to  support  the  system/ 
equipment  and  for  any  transportation  packaging  and  materials 
handling  requirements: 

(1)  Chapter  4.  Provisioning  Policy  and  Procedures,  des¬ 
cribes  the  responsibilities  for  the  provisioning  of  initial  repair 
parts.  Standard  DOD  provisioning  policies  and  procedures 
are  specified  by  reference.  The  chapter  applies  to  repair  parts 
for  the  specified  system,  and  also  to  support  equipment  (SE). 
contractor-fumished  equipment  iCFE).  and  training  equip¬ 
ment. 

121  Chapter  5,  Support  Equipment  (SE).  describes  the 
procedures,  terms,  and  conditions  governing  the  identifica¬ 
tion.  design,  and  acquisition  of  end  items  of  SE  throughout  the 
system/equipment  life  cycle  Responsibilities  for  managing 
the  acquisition  of  SE  are  specified.  Contract  requirements  and 
related  DIDS  supplement  the  policies  and  procedures 

13)  Chapters. Government  Furnished  Equipment  iGFE). 
has  the  policies,  procedures,  and  responsibilities  for  managing 
and  processing  GFE.  Standard  GFE  milestones  are  included 
along  with  a  procedural  flow  chan.  Applicable  DIDs  cover  the 
delivery  schedule  of  GFE.  delivery  status,  and  rejection  failure 
data. 

14)  Chapter  7,  Inventory  Management  Procedures. 
Policies  and  procedures  are  included  by  reference  to  the  appro¬ 
priate  DOD  documents. 

(5)  Chapter  8.  Packaging/Handling  Storage/Transporta- 
bility/Trsnsportation.  addresses  the  policy  and  procedures 
for  transportation,  packaging,  transportability,  and  related 
materials  handling  requirements  during  all  phases  of  the 
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system/ equipment  til*  cycle.  Contract  requirement!  end  stan¬ 
dard  DIDS  supplement  the  policy  and  procedures. 

16)  Chapters.  Facilities  Determination  and  Planning.  The 
funding  of  facilities  is  normally  outside  the  main  system  fund¬ 
ing  process,  but  the  provision  of  facilities  has  high  cost  poten¬ 
tial.  and  this  chapter  pro- 
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ndes  a  standi-dized  approac.i  to  identifying.  defining, 
and  planning  for  facility  requirement!.  Contract  re¬ 
quirements  and  related  data  item  description*  are 
provided. 

c  SISMS  recognizes  the  maintenance  capabilities 
of  the  services  at  all  ecnelons.  the  dynamic  nature  of 
system  equipment  repair  requirement*,  and  the  flex¬ 
ibility  that  management  must  have  to  meet  those  re¬ 
quirements.  Chapters  10.  11.  and  12  address  three  ar¬ 
ea*  which  are  important  adjuncts  to  th ;  maintenance 
process:  Preoperaaonal  support,  contractor  en¬ 
gineering  and  technical  services,  and  interservice  de¬ 
pot  maintenance.  They  provide  flexibility  to  the  logis¬ 
tics  manager  in  arriving  at  more  effective  ways  to 
maintain  and  increase  system  equipment  readiness. 

(1)  Chapter  10.  Preoperational  Support,  provides 
th*  services  and  the  contractor  with  the  policy  and  in¬ 
structions  for  fulfilling  system  support  requirements 
prior  to  the  system  entering  the  operations!  phase  of 
th*  life  cycle.  This  includes  preparation  of  recommen¬ 
ded  repair  part*  list  (RRPLl.  SE  requirements,  and  a 
support  material  list  <SML>. 

(2)  Chapter  11.  Contractor  Engineering  and 
Technical  Services  (CETS).  will  be  obtained  through 
th*  application  of  standard  policies  and  procedures  de¬ 
scribed  therein.  The  objectives,  limitations,  and  pro¬ 
curement  policies  are  discussed,  and  included  in  a  de¬ 
scription  of  recommended  Standard  Certificate  of 
Service.  A  contract  item  and  a  data  item  description 
(DID)  for  a  comprehensive  CETS  plan  are  also  part  of 
this  chapter. 

(3)  Chapter  12.  Interservice  Depot  Maintenance, 
provides  for  an  organizational  structure  in  each  of  the 
services  with  the  mission  of  enhancing  the  use  of  de¬ 
pot  resources  through  increased  service  cooperation. 
Three  level*  of  organizations  are  described: 

la)  Maintenance  interservice  support  man¬ 
agement  offices. 

(b)  Maintenance  interservice  support  plan¬ 
ning  group. 

(c)  Maintenance  interservice  support  office*. 
Policies,  procedures,  and  responsibilities  for  th*  estab¬ 
lishment  and  operanon  of  these  organization*  provide 
for  the  best  utilization  of  the  depot  maintenance  capa¬ 
bilities  of  the  services.  A  standardized  Depot  Mainte¬ 
nance  Interservice  Support  Agreement  (DMISA)  is 
included. 

d.  Chapter  13.  The  Train. ng  Program,  outlines  the 
responsibilities  for  development,  documentation,  and 
acquisition  of  the  total  training  program  by  the  execu¬ 
tive  service  including  the  support  responsibilities  of 
th*  participating  service.  It  addresses  the  essential 
phases  in  th*  development  of  a  training  program.  Con¬ 
tract  items  and  related  data  item  descriptions  support 
each  phase  of  th*  system  training  program. 

*  Chapter  14.  Configuration  Management,  pro¬ 
vides  for  the  creation  of  multi-semce  configuration 
control  boards  (CCBsi  for  jointly  used  systems/ equip¬ 
ment*  A  standard  outline  for  a  Configuration  Man¬ 
agement  Joint  Operating  Procedure  '  JOP)  covers  mul¬ 
ti-service  programs  Standard  DOD  policies  and 
procedure*  are  included  by  reference  to  th*  appropriate 
document*. 

f  Chapters  15  through  19  provide  comprehensive 
guidance  for  acquisition  and  management  of  the  vari¬ 


ous  data  required  in  support  of  system,  equipment 
programs. 

(1)  Chapter  15.  Data  Acquisition  Management, 
discusses  responsibihr.es  for  the  management  and 
control  of  data  acquisition  for  multi-service  programs 

(2)  Chapter  16.  Technical  Manual*  Acquisition 
Management,  provides  the  policy  and  procedure*  for 
implementation  of  a  technical  manual  program.  It  cov¬ 
ers  responsibilities  for  identification  of  the  overall 
technical  manual  requirement*,  preparation  of  a  Tech¬ 
nical  Manual  Plan  (TMP).  and  determination  of  qual¬ 
ity  assurance  provisions  by  in-process  review,  valida¬ 
tion,  and  verification  Contract  items  and  data  item 
descriptions  have  been  provided  to  integrate  the  sys¬ 
tem  contractor  into  this  effort. 

(3)  Chapter  17.  Engineering  Drawing*,  discusses 
current  DOD  policies  ana  specifies  application  of  stan¬ 
dardized  data  item  descriptions  for  acquisition  of  en¬ 
gineering  drawings  and  associated  lists. 

(4)  Chapter  18,  Data  Exchange  for  Product  Im¬ 
provement.  provides  for  the  exchange  of  maintenance 
and  operational  data  on  all  multi-service  programs  to 
enhance  the  decision  making  process. 

(5)  Chapter  19,  Data  Element  Dictionary.  Pro¬ 
vides  standard  data  element  definition*  by  reference 
to  MIL-STD- 1388-2. 

g.  The  following  chapters  provide  the  policies  and 
procedures  for  those  disciplines  which  support  or  sus¬ 
tain  the  other  disciplines.  The  focua  ta  oriented  to 
standardization  of  the  interfaces  between  services 
rather  than  uniformity  of  approach  within  the 
services. 

(1)  Chapter  20.  Budget  and  Funding,  provides 
the  necessary  guidance  and  reference#  to  standard 
procedures  for  the  transfer  of  fund*  for  service* 
performed. 

(2)  Chapter  21.  Procurement,  provide*  for  joint 
operation  to  ensure  appropriate  delineation  of  re¬ 
sponsibilities  between  the  executive  and  participating 
service*  consistent  with  Armed  Service*  Procurement 
Regulation  (ASPR). 

(3)  Chapter  22.  Engineering  Reeponaibility.  rec¬ 
ognizes  overall  system  engineering  as  the  re¬ 
sponsibility  of  the  executive  service.  Additionally,  it 
affords  the  participating  serviced)  an  adequate  par¬ 
ticipating  role  along  with  the  direct  engineering  re¬ 
sponsibility  for  those  portions  of  the  systems  that  are 
peculiar  to  the  participating  service!*) 

1-8.  SISMS  Muter  Flow  Chert  (attachment  3). 
Th*  SISMS  master  flow  chart  depicts  the  typical 
events  and  report  products  of  the  SISMS  activities  and 
relates  them  pictonally  to  the  system  management 
process.  The  chart  shows  how  specific  data  items  flow 
into  typical  event*  leading  to  major  decision  point*  in 
each  of  the  logistics  disciplines. 

1-9.  Procedure#  For  Changing  and  Requisi¬ 
tioning  SISMS.  AFLC  will  be  sdministrsavely  re¬ 
sponsible  for  preparation  of  all  revision*  to  this  docu¬ 
ment.  It  is  intended  that  SISMS  be  kept  dynamic. 
Changes  dictated  by  need  for  improvement  or  defi¬ 
ciencies  identified  during  implementation  and  applica¬ 
tion  should  be  documented  and  forwaraed  to  th#  office 
of  primary  responsibility  (OPR)  a*  follow#: 
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a.  DARCOM  —  Director  for  Plana.  Doctrine  and 
System*  < DROPS). 

b.  SMC  —  Deputy  Chief  Naval  Material.  Logistic 
Support  (MAT  04). 

c.  AFLC  —  Deputy  Chief  of  Staff.  Logistic*  Oper¬ 
ations  lAFLC  LO). 

d.  AFSC  —  Deputy  Chief  of  Staff,  Logistics 
(AFSC  LG). 


e.  USMC  —  Deputy  Chief  of  Staff  for  Installation 
and  Logistics  iCode  LMA). 

These  OPRs  will  assure  mterservice  consideration  and 
staffing  pnor  to  change  incorporation.  Requisition  and 
source  of  supply  will  be  through  normal  publication 
distribution  channels. 
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Chapter  5 

SUPPORT  EQUIPMENT  (SE) 


SECTION  A  -  GENERAL 
5*1.  Purpose.  This  chapter  establishes  policy  and 
procedures,  terms,  and  conditions  governing  the  iden¬ 
tification.  design,  and  acquisition  of  end  items  of  SE. 
For  multi-service  programs,  paragraph  5-4  also  serves 
as  JOA. 

5*2.  Scop*.  These  procedures  apply  to  SE  procured  to 
support  systems  equipment,  applies  to  ail  military  ser¬ 
vices.  organizations,  and  personnel  responsible  for 
planning  and  contracting  during  all  phases  of  the  sys¬ 
tem  life  cycle. 

5*3.  Policy. SE  planning  requires  support  manage¬ 
ment  attention  throughout  all  phases  of  the  life  cycle 
to  provide  positive  coordination  with  other  program 
segments.  The  total  SE  requirements  to  support  a  sys¬ 
tem  will  be  determined  from  data  items  from  the  sys¬ 
tems  contractor  in  accordance  with  procedures  set 
forth  in  Armed  Services  Procurement  Regulation 
.  (ASPRl.  and  this  chapter 

5*4.  Details: 

a.  Central  Procedures  and  Responsibilities  Basic 
factors  affecting  SE  development  of  requirements  are: 
realism  in  quantity  and  quality  of  SE  required:  consid¬ 
eration  of  locations,  pipeline,  etc.  timeliness  of  SE  ac¬ 
quisition  in  consonance  with  the  maintenance  concept, 
maintainability  analysis,  optimum  repair  level  anal¬ 
ysis.  and  LSA  These  requirements  initially  reflect 
only  the  general  equipment  required  with  provisions 
for  subsequent  progressive  development  of  specific  re¬ 
quirement  for  all  support  items.  The  SE  program  is  de¬ 
veloped  concurrently  with  the  ISP  and  is  included  as  a 
part  thereof.  The  SE  program  provisions  thereof  will 
become  part  of  the  contract  documentation. 

(It  Section  B.  as  tailored,  will  be  appended  to 
the  contract  by  the  procuring  activity  to  prescribe  the 
procedures,  terms,  and  conditions  governing  the  iden¬ 
tification.  design,  and  acquisition  of  end  items  of  SE 
required  to  support  the  end  article  on  contract.  The  ex¬ 
ecutive  service  (program  project  office)  is  responsible 
to  carefully  screen  the  requirements  of  section  B  and 
tailor  the  contents  to  their  peculiar  program  and  its 
needs. 

(2)  The  procuring  activity  will  utilize  applicable 
DIDs  in  conjunction  with  DD  Form  1423.  CDRL  to 
procure  the  necessary  data  required  to  maintain  SE 
support  for  the  end  article. 

1 3)  The  procuring  activity  will  be  responsible  for 
preparation  of  DD  Form  1423  as  outlined  in  ASPR 
and  the  timely  acquisition  of  the  data. 

(4)  Each  using  service  will  review  the  SE  recom¬ 
mendation  data  (SERD)  developed  by  the  contractor, 
evaluate  the  existing  resources,  and  determine  SE 
requirements. 

1 5 1  The  executive  service  shall  keep  the  using 
services  advised  of  anv  changes  in  the  system  pro¬ 
gramming  that  will  affect  SE  support. 


(6)  Each  using  service  will  be  responsible  for  se¬ 
lecting,  programming,  and  funding  its  own  SE 
requirements 

(71  SE  will  be  obtained  from  the  DOD  inventory 
either  through  normal  using  service  procedures  or  as 
part  of  the  prime  contract  through  the  executive 

service. 

(8)  The  procuring  activity  shall  identify  within 
the  body  of  the  basic  contract  those  specific  offices 
named  on  the  CDRL  iDD  Form  1423)  such  as  adminis¬ 
trative  contracting  officer  1 ACO).  approving  authority. 
SE  ordering  activity,  procuring  activity,  and  requiring 
activity  In  the  event  the  above  offices  are  not  identi¬ 
fied.  all  the  duties  delegated  to  them  shall  remain  the 
full  responsibility  of  the  procuring  contracting  officer 

(91  The  procuring  activity  shall  assure  that  all 
(including  computer  equipment)  software  elements  re¬ 
quired  to  operate  and  support  the  SE  are  identified,  de¬ 
veloped.  tested,  and  procured.  Precautions  must  be 
taken  to  assure  delivery  is  m  usable  format  and  with 
rights  of  data  clearly  defined  and  adequate  for  all  an¬ 
ticipated  use.  The  procuring  activity  will  assure  fi¬ 
nancial  visibility  of  costs  related  to  software  acquisi¬ 
tion  by  use  of  a  WBS  specifically  tailored  to  determine 
these  costs. 

b  Specific  Procedures  and  Responsibilities 

(1)  Upon  DOD  mulliservice  program  decision, 
the  executive  service  will  be  assigned  by  DOD 

(2)  The  executive  service  shall  convene  a  confer¬ 
ence  to  be  attended  by  SE  representatives  from  the  us¬ 
ing  services  to  establish  detailed  guidance  for  contract 
clauses.  DD  Form  1423  content,  and  the  solicitation 
(Request  for  Quotation  iRFQ).  Request  for  Proposal 
(RFP).  or  Invitation  for  Bid  (IFB>>  requirements. 

(3)  The  executive  service  will  issue  the 
solicitation. 

'4)  Offerers  will  submit  the  SE  plan  with  the  re¬ 
sponse  to  RFQ  RFP  1FB 

(5)  The  executive  service  will  establish  a  Source 
Selection  Evaluation  Board  SSEB)  in  accordance  with 
established  procedures.  Using  service  representatives 
will  be  members  of  the  Board.  Contractor  proposals, 
including  the  SE  plan  will  be  evaluated  by  the  SSEB 
and  their  recommendations  forwarded  for  final  selec¬ 
tion  through  standard  selection  procedures. 

(6)  The  executive  service  will  convene  a  precon¬ 
tract  guidance  conference  'optional  at  the  request  of 
the  contractor). 

(7)  The  executive  service  will  award  the  system 
contract 

(8)  The  executive  service  may  convene  a  post¬ 
contract  guidance  conference  ( optional  1 

(9)  Participating  services  will  review  and  an¬ 
notate  SERDs.  Annotated  SERDs  reflecting  required 
participating  service  information  will  be  forwarded  to 
the  executive  service. 
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( 10>  The  executive  service  will  resolve  techn.cal 
problems  and  all  other  conflicting  information  entered 
on  the  SERDs  prior  to  the  SE  selection  meeting  The 
resulting  consolidated  requirements  wiil  be  forwarded 
with  copies  of  each  using  service  s  SERDs  to  the  order¬ 
ing  activity 

(111  The  ordering  activity  will  furnish  one  copy 
of  each  using  service's  SERD  to  the  Resident  Inte¬ 
grated  Logistics  Support  Activity  1RILSA1  and  one 
copy  to  the  contractor  along  with  the  consolidated  or¬ 
der  for  SE  effort  and-  or  hardware. 

SECTION  B  -  CONTRACT  ITEMS 
3-3.  General  Provisions: 

a.  Purpose.  This  section  prescribes  the  procedures. 


terms,  and  conditions  goverr.inp  the  design,  identi¬ 
fication.  selection  test,  acceptance  acquisition,  and  lo¬ 
gistics  support,  of  end  items  of  SE  and  ail  computer 
Equipments  required  to  support  the  end  article  on 
contract. 

b  Application: 

(11  Incorporation.  Portions  of  this  section  shall 
be  incorporated  where  applicable,  by  reference  or  ap¬ 
pendage  to  contracts  executed  on  behalf  of  the  United 
States  Government  which  provide  for  identification  of 
SE  and/or  the  delivery  of  an  article  and  SE. 

(2)  Reference  Publications.  In  addition  to  this 
regulation,  the  following  documents,  of  the  issue  in  ef¬ 
fect  on  the  date  of  the  solicitation,  form  a  part  of  this 
requirement  to  the  extent  specified  herein. 


DEPARTMENT  OF  DEFENSE 

Cataloging  Handbooks  H4-1  and  H4-2  Federal  Supply  Code  for  Manufacturers 

Cataloging  Handbook  H6-1.  Part  1  Federal  Item  Identification  Guides  Supply  Cataloging 

DOD  4100.J8M  Provisioning  &  Other  Preprocurement  Screening  Manual. 


FED-STD-3 


MIL-C-4I30 
MIL-C-45662 
MIL-D-iOOO 
MIL-E-38793 
MIL-E-80106 
MIL-HDBK  300 
MIL-M9368 
MIL-N- 18.307 


MIL-P9024 

MIL-Q-9838 
MIL-R  9441 
MILS- i944 
MIL-S-3512 

MIL  STD-12 


MIL-STD-100 

MIL-STD-1% 

MIL-STD-470 

MIL-STD-480 

MIL-STD-785 

MIL-STD-810 

MIL-STD-864 

MIL-STD-8421 

MIL-STD-873 

MIL-STD-1388 

MIL-STD-I472 

MIL-5TD-1552 


MIL  STD  1361 
MILT-28800A 


MILITARY 

Standard  Guides  for  Preparation  of  Item  Descriptions  by  Government 
Suppliers 

Case.  Carrying.  Water-Vapor-proof 

Calibration  System  Requirements 

Drawings.  Engineering  and  Associated  Lusts. 

Manuals.  Calibration  Procedure. 

Engineering  Services.  Calibration  Equipment. 

Technical  Information  File  of  Support  Equipment. 

Microfilming  of  Engineering  Document*.  35mm;  Raquirements  for. 

Nomenclature  and  Nameplate*  for  Aeronautical  Electronic  and  Asaoci- 
ated  Equipment. 

Packaging.  Handling,  and  transportation  in  system  equipment 
acquisition. 

Quality  Program  Requirements 

Record.  Aircraft  Inventory 

Slings.  Aircraft:  General  Specification  for 

Support  Equipment.  Aeronautical.  Special.  General  Specification  for 
Design  of. 

Abbreviations  for  Use  on  Drawings  and  in  Technical  Type 
Publications 

Engineering  Drawing  Practices. 

Joint  Electronics  Type  Designation  System.  f 

Maintainability  Program  Requirements  for  Systems  and  Equipments. 

Configuration  Control  —  Engineering  Changes.  Deviations,  and 
Waivers. 

Requirements  for  Reliability  Program  for  Systems  and  Equipments. 

Environmental  Test  Methods. 

Ground  Support  Equipment  Functional  Classification  Categories. 

Air  Transport  ability  —  General  Requirement#  for 

Type  Designation  System  for  Aeronautical  and  Aeronautical  Support 
Equipment. 

Logistic  Support  Analysis 

Human  Engineering  Design  Criteria  for  Military  Systems  Equipment 
and  Facilities 

Provisioning  Technical  Documentation,  Uniform  DOD  Requirements 
for 

f  i”isioning  Procedures.  Uniform  DOD 

-.  Equipment  for  Use  with  Electrical  and  Electronic  Equipment; 
General  Specification  for 
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commercial 

ANSI  Y52.16a  Reference  Designation  for  Electrical  and  Electronic  Part*  and 

Equipment. 

ANSI  Y32.2  Graphic  Symbols  for  Electrical  and  Electronic  Diagrams 

3i  Other  Documents 


DID  SUMBER 

DI-A-6102 

DIE-6120 

DI-E-6121 

DI-F-6127 

DIM-6153 

DIP  6163 

Dl-S-6171 

DI-S-61*6 

DI-S-6177 

Dl-V-6183 

DI-V-6184 

DI-V-6185 

DI-V-6186 

DI-V-7016 

D I-S- 70 17 


DA  TA  ITEM  DESCRIPTION'  ‘DID/ 

Tirr.E 

Support  Equipment  Plan  (SEP) 

Support  Equipment  Illustrations  >SET) 

Support  Equipment  Installation  Data 
Support  Equipment  End  Item  Funding  Report 
Technical  Manuals  Commercial  Literature 
Support  Equipment  Delivery  Schedule  Delinquency  Report 
Logistic  Support  Analysis  Record  iLSAR) 

Support  Equipment  Recommendation  DataiSERD) 
Calibration  Measurement  Requirements  Summary 
Consolidated  Support  Equipment  List  iCSEL) 

Proposed  Revision  to  the  Support  Equipment  Exhibit 
List  of  Standard  Modified  Hand  Tool* 

Priced  Support  Equipment  List 

Provisioning  and  Other  Preprocurement  Screening  Data 
Logistics  Support  Analysis  Plan 


c  Time  Cycles  Unless  otherwise  specified,  the  time 
cycles  and  limitations  established  by  this  document 
are  maximum*  When  nei-essarv  to  provide  timely  sup¬ 
port  by  operational  need  dates  different  time  sched¬ 
ules  will  be  mutuallv  established  at  the  guidance  con¬ 
ference  see  paragraph  o-6bi 

d.  Subcontracts  The  contractor  shall  include  appli¬ 
cable  requirements  of  this  chapter  in  ail  contracts 
with  vendors  for  articles  requiring  identification  of 
SE.  He  snail  include  the  programming  information 
with  the  purchase  order  contract  as  appropnate  or  as 
soon  as  such  data  are  made  available  to  him  by  the 
Government. 

e  Follou On  Contracts  Procurement  documents 
for  follow-on  contracts  shall  not  require  duplication  of 
data  except  as  provided  in  the  appropnate  DID 

f  Repair  Parts  for  SE  Repair  pans  in  support  of 
SE  procured  on  this  contract  will  be  documented  and 
furnished  in  accordance  with  procedure*  set  forth  in 
the  contract.  The  requirement  for  repair  parts  docu¬ 
mentation  for  the  SE  item  will  be  annotated  on  the 
SERD. 

g.  Vendor  Items.  The  Government  may  at  its  op¬ 
tion  and  after  coordination  with  the  prime  contractor, 
effect  direct  contact  with  the  prime  contractor  s  ven¬ 
dors  for  the  purpose  of  review  familiarization,  and  co¬ 
ordination  on  the  selection  of  SE  Throughout  the  di¬ 
rect  contact,  actions  shall  not  he  taken  that  would 
infer  or  imply  a  commitment  on  the  part  of  the  Gov¬ 
ernment  or  prime  contractor  Any  item  selected  for 
subsequent  procurement  at  the  time  of  review  shall 
not  be  considered  bv  the  vendor  to  constitute  a  com¬ 
mitment  or  obligation  on  the  part  of  the  Government 
or  prime  contractor  to  order  the  selected  items.  Prime 
contractors  and  their  vendors  will  further  ensure  that 
any  contract  entered  into  with  vendors  will  include  an 
appropnate  clause  wherenv  the  vendor  agrees  to  fur 


nish  directly  to  the  Government  information  copies  of 
such  documentation  as  is  prescribed  in  this  document 
and  give  such  assistance  as  may  be  necessary  to  en¬ 
able  the  Government  to  select  SE. 

h  Automatic  Test  Equipment  'ATE  The  con¬ 
tractor  shall  insure  that  computer  equipment  used  as 
SE  'programmable  test  equipment!  are  planned,  devel¬ 
oped  i  using  an  approved  computer  language),  ac¬ 
quired.  employed  and  supported  to  effectively,  effi¬ 
ciently.  and  economically  accomplish  the  assigned 
mission  of  the  participating  service  Such  effort  shall 
be  in  accordance  with  applicable  contractual 
requirements 

5-6.  Initial  Information  and  Guidance: 

a  Initial  Information.  With  the  executed  copy  of 
the  contract,  the  procuring  activity  shall  normally,  on 
behalf  of  the  United  States  Government,  furnish  the 
contractor  complete  program  data  on  the  employment 
and  deployment  of  the  end  article  on  contract.  Such 
information  shall  be  revised  as  required  to  provide  the 
contractor  with  the  latest  information  on  planned  em¬ 
ployment  and  materiel  support  for  the  end  article  on 
contract  In  the  event  this  is  not  furnished  with  the 
contract  such  information  shall  be  furnished  to  the 
contractor  at  the  time  of  the  guidance  conference, 
b.  Guidance  Conference . 

(1)  Precontract  Guidance  Conference.  A  request 
for  guidance  information  may  be  made  by  an  offerer 
as  part  of  the  precontraetuai  source  selection  pro¬ 
ceedings.  Specific  questions  and  or  a  proposed  agenda 
of  s  guidance  meeting  shall  be  included  in  this  re¬ 
quest.  The  conference  will  be  held  within  15  days  after 
receipt  of  an  offer  s  request.  At  this  conference,  the 
procuring  acu  nty  will  provide  complete  instructions  to 
all  participating  offerers  regarding  the  extent  of  work 
to  be  performed  under  this  document,  suomittsu  re- 
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quiremenu.  approval  times,  applicability  of  specifica¬ 
tions  and  standards  other  than  those  cited  herein,  and 
appropriate  data  requirements,  especially  the  SEP 
(Support  Equipment  Plam,  which  outlines  the  con¬ 
tractor  s  plan  for  the  5E  program  for  an  article,  will 
be  addressed. 

<2)  Postcontract  Guidance  Conference  The  con¬ 
tractor  shall  recommend  to  the  procuring  activity  a 
date  for  convening  a  postcontract  guidance  confer¬ 
ence.  Specific  questions  and.  or  a  proposed  agenda  of 
the  guidance  meeting  shall  be  included  in  this  request 
The  dace  recommended  shall  be  at  the  earliest  possible 
mutually  acceptable  date  but  not  later  than  thirty  (30) 
days  after  award  of  the  initial  conaact.  The  purpose 
of  this  conference  will  be  to  provide  the  consactor 
with  policy  direction  and  guidance  concerning  the  re¬ 
quirements  of  this  document  and  applicable  DIDs. 

(3)  Supplemental  Conferences.  When  it  is  deter¬ 
mined  that  subsequent  conferences  must  be  held,  the 
procuring  activity  and  the  contractor  shall  establish 
menially  acceptable  dates  for  convening  such  confer¬ 
ee.  a.  When  required,  they  shall  be  convened  at  the 
can  of  the  procuring  activity 

14)  Establishment  of  Guidance  Conferences.  The 
procuring  activity  shall  establish  a  firm  date  for  the 
conferences.  The  conference  team  shall  be  composed  of 
responsible  Government  and  offer  contractor  repres¬ 
entatives.  The  conference  shall  be  chaired  by  a  repres¬ 
entative  of  the  procuring  activity 

c.  Description  of  Procedures.  The  following  pro¬ 
cedures  will  be  utilized: 

<l)  Conference.  Guidance  conference  occur  be¬ 
tween  Government  and  the  contractor  as  outlined  in 
this  document. 

(2)  SE  Plan  Submissions.  The  contractor  shall 
prepare  and  submit,  in  accordance  with  the  CDRL  re¬ 
quirements.  an  SE  Plan  iDl-A-6102)  Rcsubmissions  of 
the  plan  shall  be  accomplished  throughout  the  life  of 
the  contract  to  expand  the  content  of  the  document  to 
include  and  update  all  information  required  by  the 
DID 

(3)  SERD  Submissions  The  contractor  shall  pre¬ 
pare  and  submit  a  SERD  'DID  DI-S-61T6)  for  each 
item  of  SE  required  to  satisfy  functional  requirements 
identified.  SERD  submissions  are  a  continuing  re¬ 
quirement  throughout  the  life  of  the  contract.  SE  iden¬ 
tified  during  the  program  initiation  or  prototype  con¬ 
tractual  efforts  shall  be  reflected  in  the  full  scale 
development/ production  contract.  For  those  items  and 
quantities  not  ordered  in  the  program  initiations/  pro¬ 
totype/ full  scale  development  contracts,  the  ordering 
activity  may  authonze  the  contractor  to  proceed  with 
the  design,  development,  and  manufacture  or  pro¬ 
curement  portion  of  the  program  under  the  production 
contract  For  production  contracts  on  weapon  systems 
or  subsystems  which  have  been  covered  by  SERDs 
during  previous  development  or  production  contracts 
for  the  same  end  article,  resubmission  of  SERDs  are 
not  required  unless  new  requirements  are  identified. 
Where  one  SE  item  may  satisfy  two  or  more  different 
functional  requirements,  and  the  previous  SERD  sub¬ 
mittal  does  not  reflect  this  fact,  resubmittai  of  the 
SERD  using  the  same  item  number  is  required  indi¬ 
cating  the  new  -  additional  information 

<ai  Selection  and  Description  of  Items  Selec 
non  end  description  of  items  shall  conform  to  the  fol¬ 
lowing  requirements  To  promote  standardization  of 


SE  within  the  Government,  the  contractor  shall  con¬ 
sider  the  following  order  of  priority  in  preparing 
recommendations: 

(1)  FIRST  Equipment  defined  by  current 
Government  specifications  or  moamcation  of  such 
equipment 

(2)  SECOND:  Off-the  shelf  commercial 
equipment  currently  in  the  Government  inventory  for 
which  procurement  data  are  available. 

(3)  THIRD:  Other  off-the-shelf  commercial 
or  modified  commercial  equipment 

(4)  FOURTH:  Proposed  equipment  to  be 
developed  by  the  contractor,  subcontractor,  or  associ¬ 
ate  contractor  The  contractor  shall  identify  any  items 
of  GFE  which  are  to  be  incorporated  into  the  proposed 
item  of  SE.  isee  chapter  6). 

The  contractor  in  preparing  his  recommendations 
shall  obtain  from  the  procuring  activity  the  Technical 
Information  File  (M1L-HDBK-300)  for  his  use  in  deter¬ 
mining  whether  a  SE  functional  requirement  can  be 
fulfilled  by  an  item  of  equipment  which  is  already  in 
Government  inventory.  Participating  services  lists  of 
preferred  SE  shall  also  be  consulted. 

fb>  Exclusions.  Unless  otherwise  specified  in 
the  contract.  SERDs  shall  not  be  prepared  for  the  fol¬ 
lowing  items  or  classes  of  equipment: 

(1)  Common  powered  and  non-powered 

handtools. 

(2)  Housekeeping  items 

1 3)  Office  furniture  and  similar  equipment 
which  are  required  as  indirect  support  and  are  defined 
in  applicable  allowance  lists. 

(4)  Common  production  tools  and  tooling, 
for  example,  lathes,  drill  presses,  plating  equipment, 
grinders,  induction  heaters,  etc. 

1 5)  Items  which  are  used  only  by  the 

contractor. 

(61  Personal  equipment,  for  example, 
headsets,  microphones,  etc 

(7)  Offline  ADP  equipment. 
ic>  Initial  SERD  Submissions  Initial  SERDs 
shall  be  submitted  by  the  contractor  not  later  than  30 
days  or  as  otherwise  authorized  after  receipt  of  the 
funding  authorization  The  procuring  activity  may  af¬ 
ter  review  of  the  initial  submission  (or  as  might  be¬ 
come  necessary  at  any  subsequent  time),  alter  the  rec¬ 
ommendation  The  description  of  an  item  either 
procurer" ent  or  engineering  critical  will  actuate  a  close 
liaison  between  agencies  designated  by  the  system 
program  director  or  appropriate  Government  manage¬ 
ment  agency,  ensuring  effective  followup  of  the  item  of 
SE  involved  in  accordance  with  programming 
requirem  ents. 

(d)  Subsequent  SERD  Submission.  Unless 
otherwise  specified  in  the  contract,  subsequent  SERDs 
shall  be  submitted  on  such  a  schedule  'normally  at  30- 
day  intervals)  to  provide  the  Government  sufficient 
time  to  review  the  proposed  item  and  authonze  pro¬ 
curement  and  delivery  to  support  the  first  end  article 
delivered  to  the  Government  More  frequent  sub¬ 
missions  may  be  made  under  mitigating  circum¬ 
stances.  for  example,  engineering  procurement  critical 
or  safety  of  flight 

1  ei  SERD  Review  SERDs  will  be  revieweo  oy 
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th*  Government  and  appropnate  portions  returned  to 
the  contractor  not  later  than  60  days  after  receipt  by 
the  SE  approving  authority 

<4i  Contractor  SERD  Actions  Upon  receipt  of 
the  approved  SERD  from  the  Government,  the  con¬ 
tractor  snail,  as  authorized  by  the  ordering  activity, 
inmate  the  actions  indicated  in  the  SERD.  such  as: 

>ai  Changes  in  design  characteristics,  if  indi¬ 
cated  on  Figure  la  of  the  SERD. 

lb>  Establish  procedures  to  ensure  continuous 
and  close  liaison  with  the  Government  during  the  de¬ 
velopment.  procurement  of  designated  SE  critical 
items. 

(c)  Prepare  end  item  specifications,  or  other 
specifications,  in  accordance  with  the  contract 
requirements. 

(d)  Prepare  SEI  in  accordance  with  DID  DI¬ 
E-6120. 

lei  Prepare  technical  manuals  in  accordance 
with  DID  DI-M-6153. 

<0  Prepare  Maintenance  requirements  cards 
in  accordance  with  the  contract  requirements 

ig)  Prepare  CMRS  ion  system  component 
and/or  SE  item)  in  accordance  with  DID  DI-S-6177 

(h)  Prepare  Installation  data  in  accordance 
with  DID  DI-E-6121. 

(i)  Prepare  repair  parts  provisioning  tech¬ 
nical  documentation  in  accordance  with  MIL-STD- 
1552  isee  chapter  4). 

<j>  Proceed  with  design  and  fabncauon  pro¬ 
curement  in  quantities  indicated  on  figure  lb  of  the 
SERD 

(k>  Compile  data  for  periodic  submission  of 
the  CSEL  in  accordance  with  DrD  DI-V-6183. 

5-7.  Data  Requirements:  Types  of  Data.  As  speci¬ 
fied  on  the  contract  data  requirements  list  <DD  Form 
1423).  the  following  SIS.MS  data  item  descriptions  ex¬ 
plain  types  of  data  to  be  provided: 

a.  SE  Recommendation  Data  'SERDi  The  con¬ 
tractor  shall  prepare  and  submit  SERDs  in  accordance 
with  DID  DI-S6176. 

b  Consolidated  SE  List  'CSELj  The  contractor 
shall  prepare  and  submit  CSEL  in  accordance  with 
DID  DI-V-6183. 

<1)  Initial.  Not  later  than  thirty  ( 30)  days  after 
receipt  of  approved  SERDs  from  the  Government,  the 
contractor  shall  furnish  a  CSEL  reflecting  all  items  of 
SE  selected  as  well  as  those  not  selected. 

<2)  CSEL  Revisions  At  60-day  intervals,  the 
contractor  shall  prepare  revised  or  additional  pages  re¬ 
flecting  changes  to  items  previously  listed,  or  the  addi¬ 
tion  of  new  items,  for  which  the  contractor  has  re¬ 
ceived  an  annotated  SERD  Government  decisions  will 
continue  to  be  reflected  on  the  CSEL  All  CSEL  re¬ 
visions  shall  include  change  sheets  to  the  Revision  In¬ 
dex  (identified  as  Sequence  3).  Revision  sheets  for  Se¬ 
quences  3  through  10.  as  applicable,  shall  be 
submitted  whenever  more  than  ten  items  are  involved 
in  the  revisions.  The  revision  submission  which  coin¬ 
cides  with  the  semiannual  CSEL  reissue  referred  to  in 
paragraph  5-7b«3i,  shall  be  incorporated  as  part  of  the 
reissue.  If  no  approved  disapproved  SERDs  have  been 
received  from  the  Government  within  the  last  60-day 


period,  a  negative  report  will  be  submitted  by  tne 
contractor. 

1 3)  CSEL  Reissues.  Unless  otherwise  specified, 
the  contractor  shall  upcate  and  reissue  the  CSEL  ev¬ 
ery  180  days  for  the  purpose  of  incorporating  the  re¬ 
visions  submitted  in  accordance  with  paragraph  5- 
Tb(2>.  The  reissue  precludes  a  requirement  for  sub¬ 
mission  of  concurrent  revision.  If  no  revisions  have 
been  effected  against  the  CSEL  during  the  180-day  re¬ 
issue  cycle,  no  reissue  is  required  and  a  negative  re¬ 
port  shall  be  submitted  All  revisions  shall  be  refer¬ 
enced  to  permit  ready  identification.  A  reissue  shall 
always  be  submitted  within  30  days  after  award  of  a 
follow-on  contract  for  additional  end  articles.  This  re¬ 
issue  establishes  a  new  cycle  for  the  issuance  of  re¬ 
visions  and  reissues. 

(4)  CSEL  for  SE  Selection  Review  Meetings.  In 
those  instances  when  a  SE  selection  review  meeting 
must  be  held  (paragraph  5-9d>.  the  contractor  shall 
make  available  a  CSEL  Reissue  for  the  use  of  ail  at¬ 
tendees  at  the  meeting.  This  CSEL  Reissue  shall  in¬ 
clude  ail  items  of  SE  previously  submitted  by  the  con¬ 
tractor  and  approved,  disapproved  by  the  Government. 
The  contractor  shall  identify  the  list  in  the  heading  as 
the  ‘Consolidated  Support  Equipment  Lust  for  SE  Se¬ 
lection  Review  Meeting  '  The  contractor  shall  provide 
copies  of  the  list  as  specified  "by  the  Government. 

c.  CMRS  ’ Calibration  Measurement <  Requirements 
Summary /  The  C MRS  jhail  be  prepared  in  accord¬ 
ance  with  DID  DI-S-6177  and  as  specifically  author¬ 
ized  in  the  SERD  by  the  Government.  Initial  sub¬ 
mission  of  the  CMRS  shall  be  not  later  than  120  days 
after  funding  authorization  for  required  effort.  Re¬ 
visions  shall  be  submitted  progressively,  within  sys¬ 
tem  (Category  li  or  as  SE  measurement  requirements 
become  know 

d.  SE  Illustrations  'SEh.  The  contractor  shall  pre¬ 
pare  SEI  in  accordance  with  DI-E-6120.  but  only  for 
those  items  specified  by  the  Government  on  the  ap¬ 
proved  SERD.  SEI  data  shall  not  normally  include 
common  tools  found  in  the  standard  machine  shop, 
component  parts  or  subassemblies  of  SE  end  items, 
kits,  sets  of  tools,  fixtures  for  manufacturing  or  depot 
maintenance  use.  slings  adapters,  small  containers, 
cabinets,  or  items  of  supply  with  status  classification 
of  "obsolete.  ' 

( 1 1  SEI  Acceptance  SEI  furnished  in  accord¬ 
ance  with  this  requirement  shall  be  subjected  to  in¬ 
spection  and  approval  in  accordance  with  applicable 
specifications. 

(2)  Preparation  for  Delivery  isee  chapter  8): 
i a)  Packaging  The  DD  Forms  1736  and  illus¬ 
trations  shall  be  packaged  flat  between  two  pieces  of 
domestic  fiberboard. 

(b)  Packing.  Unless  otherwise  specified,  the 
product  shall  be  packed  in  containers  which  (together 
with  packaging  and  wrapping)  are  acceptable  by  com¬ 
mon  earners  for  safe  transportation. 

ic)  Marking  The  following  information  shall 
appear  on  the  outside  of  each  shipping  container: 
Support  Equipment  Illustrations 

Name  of  Contractor _ 

Contract  or  Order  Number _ 

End  Article  System  Application  _ 
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e.  SEDSDR  'SE  Delivery  Schedule  Delinquency 
Report)  Thu  report  shall  be  prepared  in  accordance 
vith  DID  DI-P-6165. 

f.  Revisions.  Revisions  to  the  approved  delivery 
schedule  shall  only  be  submitted  when: 

•  1)  New  SE  items  are  added  isame  time  frame 
as  above). 

1 2)  Quantities  are  revised  (same  time  frame  as 
above). 

1 3)  Any  item  is  anticipated  to  become  delinquent 
las  soon  as  known). 

(4)  Any  Item  has  actually  become  delinquent 
iwithin  5  days  after  the  actual  delinquency  and  con¬ 
tinuing  at  the  30-day  intervals  until  there  are  no  delin¬ 
quent  items). 

g  LSMHT  'List  of  Standard  Modified  Hand 
Tools/.  This  list  shall  be  prepared  in  accordance  with 
DID  DI-V-6185  and  submitted  not  later  than  120  days 
pnor  to  delivery  of  the  first  end  article  on  the  contract. 
Revisions  tr  the  list  shall  be  submitted  as  required 

h.  PSEL  < Priced  SE  List /  and  SE  Exhibit  The 
PSEL  shall  be  provided  in  accordance  with  DID  DI-V- 
6138  and  procedures  indicated  in  paragraph  3-12a.  The 
approved  and  agreed  upon  PSEL  with  cancellation 
addendum,  if  any)  results  in  the  SE  Exhibit  Applica¬ 
ble  procedures  are  indicated  in  paragraph  5- 12b  Re¬ 
visions  to  the  SE  Exhibit  will  be  provided  in  accord 
ance  with  DID  DI-V-6134  and  procedures  indicated  in 
paragraph  3- 12b. 

i  SE1FR  SE  End  Item  Funding  Report,  This  re¬ 
port  shall  be  prepared  in  accordance  with  DID  DI  F 
6127  and  procedures  indicated  in  paragraph  5-13  The 
report  reflects  the  funding  status  of  each  SE  contract 
line  item,  as  well  as  each  different  funding  citation,  in 
the  end  article  contract.  Thu  report  shall  be  submitted 
by  the  10th  day  of  each  month. 

j  SEP  'SE  Plan/  To  be  provided  in  accordance 
with  DID  DI-A-6102. 

il)  Plan.  Initial  Submission.  Concurrently  with 
and  as  a  part  of  the  contractor  s  proposal  quotation 
for  the  end  article  program 

'2)  Plan  Revisions  During  negotiation  of  the 
end  article  contract  and  as  required  to  reflect  end  arti 
cie  or  SE  program  changes.  Revisions  also  may  result 
from  the  guidance  conferences  convened  pursuant  to 
paragraph  5-6b 

k.  SEID  'SE  Installation  Data/.  SEID  shall  be  pre¬ 
pared  in  accordance  with  DID  DI  E  6121  as  authorized 
by  the  Government  in  approved  SERDs  for  end  items 
of  SE  requiring  installation  in  existing  or  planned 
shipboard  or  shore-based  facilities.  Initial  SEID  shall 
be  submitted  not  later  than  30  days  after  release  of  the 
SE  item  for  fabrication.  Revisions  shall  be  submitted 
within  30  days  after  release  of  any  change  in 
fabrication. 

l.  S£  nomenclature  To  be  provided  in  accordance 
with  Cataloging  Handbook  H6-1 

m  Data  The  following  types  of  data  are  to  be 
provided 

'll  SE  Engineering  Data  Drawings  and  Associ¬ 
ated  Lists  —  In  accordance  with  contract 

requirements 

'2)  SE  Preservation  and  Packaging  —  in  accord 
ance  with  contract  requirements 


■3)  SE  Logistics  Support  Analyses  —  The  con¬ 
tractor  shaii  prepare  a  LSA  for  items  of  SE  as  ..nai- 
cated  in  the  Government  approvea  SERD  The  anal¬ 
ysis  will  be  specifically  maicatea  to  the  contractor, 
and  may  be  selected  from  DIDs  DI-5-7017  or  DI-S- 
6171. 

14)  SE  Technical  Manuals  —  In  accordance  with 
DID  established  by  the  procunng  activity 

15)  SE  Repair  Parts  Provisioning  Documen- 
tation  —  The  contractor  shall  prepare  and  submit  to 
the  Government.  SE  repair  parts  provisioning  docu¬ 
mentation/  lists  for  each  SE  ena  item  specifically  au¬ 
thorized  on  approved  SERDs.  The  provisioning  docu¬ 
mentation/  lists  shall  be  prepared  in  accordance  with 
MIL-STD- 1552  (see  chapter  41 

(6)  SE  Provisioning  Screening.  The  contractor 
shall  perform  SE  provisioning  screening  for  eacr.  SE 
end  item  pnor  to  submission  of  the  SERD  Sucn  pro¬ 
visioning  screening  data  shall  be  prepared  in  accord¬ 
ance  with  DID  DI-V  7016. 

1 7)  SE  item  Description.  The  contractor  shall 
prepare  and  submit  SE  item  descriptions  for  eacn  SE 
end  item  specifically  authonzed  on  approved  SERDs 
The  SE  item  descnptions  shall  be  prepared  ana  sub¬ 
mitted  in  accordance  with  FED-STD-5. 

5-8.  SE  Identification  and  Design: 

a  Funding  Requirements  ana  Authorization  Prior 
to  execution  of  the  contract,  the  cont.-u.  ;or  shall  id 
vise  the  SE  approving  authority  througn  '.rtc  P1"1'1  ::  c 
estimated  tunas  required  for  nonrecurring  engineering 
effort  Additional  funding  requirements  si. a.,  be 
shown  separately  and  indicate  the  period  .0  .  m  v-r-ra 

for  SE  design  effort.  SE  sustaining  engineering  effort, 
and  cost  of  preparing  each  data  item  estimatea  by  t.ne 
contractor  to  be  required  herein  and  prescribed  in 
paragraph  5-7  The  SE  Approving  Authority  will,  after 
receipt  of  the  funding  estimate  and  contract  award,  is¬ 
sue.  or  cause  to  be  issued,  appropriate  funaing  author¬ 
ization  documents  to  cover  the  services  and  data  to  be 
authonzed  or  advise  the  contractor  tnat  effort  ;s  not 
required  under  this  contract  item 
b  SE  Identincation 

■I)  End  Article  Analysis  The  contractor  shall 
investigate  requirements  for  SE.  as  substantiated  by 
ena  article  LSA  in  accordance  with  DI-S-'Ol"  see 
chapters  J  and  61  when  specified  in  the  contract  If  SE 
design. and  procurement  manufacturing  leadtime  con¬ 
siderations  dictate  recommending  the  SE  item  to  the 
AA  in  order  to  meet  test  operational  requirements  be¬ 
fore  the  LSA  has  been  formally  documented,  t.ne  lead- 
time  shall  dictate  such  submission.  Upon  completion 
of  the  LSA.  the  contractor  shall  immediately  rev  ise  his 
SE  submission  if  the  analysis  reveals  changed  SE 
requirements 

1 2)  SE  for  Government  Furnished  Components. 
Normally.  SE  strictly  related  to  a  Government-fur 
nished  component  installed  in  the  end  article  is  ex 
eluded.  However  items  that  are  required,  due  to  pecu 
iiarities  of  the  installation  of  the  Government 
furnished  component  in  the  end  article,  are  requirea  to 
hr-  investigated  and  submitted  for  consideration 

ti  Selection  Criteria  To  promote  stanaard 
izatiun  0!  SE  within  the  Government  the  contractor 
snail  onsioer  the  following  oraer  of  priontv  :n  pre¬ 
paring  recommendations  First,  eouipment  aetinec  nv 
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current  Government  specification  or  modification  of 
such  equipment,  second,  private!;'  developed  commer¬ 
cial  equipment  currently  in  the  Government  inventory 
which  have  been  qualified  to  the  requirement  and  for 
wmch  procurement  data  are  available,  third,  other 
such  pnvateiy  developed  commercial  or  such  modified 
equipment  which  can  be  qualified.  Fourth,  proposed 
equipment  to  be  developed  by  the  contractor,  sub-con¬ 
tractor  or  associated  contractor. 

14)  Government  Inventory  Information.  The  con¬ 
tractor  in  preparing  his  recommendations,  shall  ob¬ 
tain  from  the  procuring  activity  the  Technical  Infor¬ 


mation  File  MIL  Handbook-3001  for  his  use  in 
determining  whether  a  SE  functional  requirement  can 
be  fulfilled  by  an  item  of  equipment  wmch  is  aireadv 
in  Government  inventory  Participating  service  lists  of 
preferred  SE  shall  also  be  consulted. 

c.  SE  Design.  SE  design,  drawings,  and  detailed 
specifications  shall  be  in  accordance  with  the  follow¬ 
ing  specifications,  as  appropriate  to  the  nature  of  the 
particular  item,  except  that  items  required  solely  for 
depot  level  use  may  be  designed,  and  drawings  pre¬ 
pared  in  accordance  with  the  manufacturer  s  normal 
commercial  practices. 


MIL-C-4150 
MIL-D-1000 
MIL-N- 18307 


MIL-S-S944 

MIL-S-S512 

MIL  STD- 12 

MIL-STD-100 

MIL-STD-196 

MIL-STD-470 

MIL-STD-785 

MIL-STD-810 

MIL-STD-875 


MIL- STD-882 
MIL-STD-1472 


MIL-T-28800A 


ANSI  Y32.I6a 
ANSI  Y3Z2 


Ca»e.  Carrying.  Water- Vapor-proof 

Drawings.  Engineering  and  Associated  Lists 

Nomenclature  and  Nameplate  for  Aeronautical  Electronic  and  Associ¬ 
ated  Equipment. 

Slings,  Aircraft.  General  Specification  for 

Support  Equipment.  Aeronautical.  Special.  General  Specification  For 
Design  of. 

Abbreviations  for  L'se  on  Drawings  and  in  Technical  Type 
Publications. 

Engineering  Drawing  Practices 

Joint  Electronics  Type  Designation  System. 

Maintainability  Program  Requirements  for  Systems  and  Equipments 

Requirements  for  Reliability  Program  for  Systems  and  Equipments 

Environmental  Test  Methods 

Type  Designation  System  for  Aeronautical  and  Aeronautical  Support 
Equipment 

System  Safety  Program 

Human  Engineering  Design  Criteria  for  Military  Systems  Equipment 
and  Facilities. 

Test  Equipment  for  Use  with  Electrical  and  Electronic  Equipment 
General  Specification  for 

Reference  Designation  for  Electrical  and  Electronic  Parts  and 
Equipment. 

Graphic  Symbols  for  Electrical  and  Electronic  Diagrams 


NOTE;  Additional  specifications  may  be  cited  in  the  contract,  as  appropriate 


d.  Use  of  the  SEED 

1 1)  Contractor  Furnished  SE.  Requirements  for 
design,  development,  fabrication,  procurement  test 
and  initial  quantities  of  SE  shall  be  established 
through  submission  and  approval  of  the  SERD 
Changes  in  design  information,  if  required,  will  be  in¬ 
dicated  on  an  approved  SERD  Figure  la  returned  to 
the  contractor  Government  decisions  on  SE  items  are 
reflected  on  an  approved  copy  of  the  SERD  Figure  lb 
returned  to  the  contractor  Information  required  in  ad¬ 
dition  to  that  contained  in  the  SERD  may  be  re¬ 
quested,  where  necessary,  by  the  SE  AA  prior  to 
granting  the  contractor  authorization  to  proceed.  The 
additional  data  my  be  in  the  format  of  an  EIS. 

NOTE:  SE  identified  during  the  validation  phase  of 
the  end  article  life  cycle  shall  be  submitted  as  SERDs 
in  the  full  scale  development  production  contract  For 
those  items  and  quantities  not  ordered  in  previous 
phases,  the  procuring  activity  may  authorize  the  con¬ 
tractor  to  proceed  with  the  design,  development  and 
manufacture  or  procurement  portion  of  the  program 
under  the  production  contract. 

(2)  Government  Furnished  SE  Approval  bv  the 
Government  of  the  entry  G"  block  12.  of  the  SERD 
indicates  that  the  Government  intends  to  provide  the 
item  vice  contractor  furnished.  Items  which  are  deter 
mined  to  be  properly  Government  furnished  but 
which  the  Government  is  unable  to  provide  on  a  time¬ 


ly  basis,  may  be  authorized  for  contractor  pro¬ 
curement  In  such  instances,  the  AA  will  provide  ap¬ 
propriate  procurement  data  and  recommended  sources. 

1 3)  Logistics  Support  for  SE  Logistics  support 
requirements  for  approved  items  of  SE  are  initiated 
through  the  submittal  and  approval  of  appropriate  in¬ 
formation  on  the  SERDs  Certain  information  per¬ 
taining  to  iogistics  support  requirements  for  individual 
SE  items  is  proposed  b>  the  contractor  on  the  SERDs. 
The  Government  approves,  revises  and  or  adds  iogis- 
tics  support  element  information  on  the  SERDs  and 
authorizes  the  contractor  through  the  ordering  activ¬ 
ity  to  initiate  required  logistics  support  action  Such 
action  includes  the  preparation  of  technical  manuals, 
maintenance  requirements  cards,  csdibration  mea¬ 
surement  requirements  summary,  provisioning  screen¬ 
ing  data,  repair  parts  provisioning  documentation, 
item  descriptions,  installation  data  and  LSA  planning 
data  when  applicable  and  authorized  by  the  Govern¬ 
ment  on  the  approved  SERD. 
e  Design  Changes 

1 1 )  Class  I  Changes.  If.  after  establishment  of  a 
firm  configuration  of  an  item  of  SE.  it  is  considered 
necessary  to  change  the  design,  eliminate,  supersede, 
or  aad  items  of  SE  because  of  any  of  the  following 
conditions,  the  contractor  shall  select  and  inmate  the 
appropriate  action  as  described  herein  A  firm  config¬ 
uration  is  estaohshed  on  the  signing  of  the  DD  Form 
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250  for  the  first  item.  After  submission  of  the  firm 
price  proposal  but  pr.or  to  signing  of  the  DD  Form  250 
for  the  first  item,  the  contractor  may  make  category  D 
or  E  changes  without  submission  of  an  ECP  provided 
these  changes  do  not  affect  or  aiter  capability  or  nego¬ 
tiated  cost  of  the  SE  item. 

(a)  CATEGORV  A. 

I  Condition:  Change  in  end  article  dic¬ 
tates  requirement  for  new  item! si  of  SE.  Does  not  af¬ 
fect  existing  items. 

2.  Action:  The  end  article  ECP  shall  be  ac¬ 
companied  by  a  SERD  for  each  new  item.  The  SERD 
will  be  processed  by  the  Government  in  conjunction 
with  ECP  processing;  action  on  each  SERD  will  be 
taken  concurrent  with  the  ECP  action  to  the 
contractor. 

lb)  CATEGORY  B: 

1.  Condition:  Change  in  end  article  dic¬ 
tates  requirement  for  new  itemisi  of  SE  which  com¬ 
pletely  supersedes  existing  itemisi.  Superseded  itemisj 
cannot  be  brought  to  configuration  of  new  itemisi. 

2.  Action:  The  end  article  ECP  shall  be  ac¬ 
companied  by  a  SERD  for  each  item.  The  SERD  will 
be  processed  by  the  Government  in  conjunction  with 
the  ECP  action  to  the  contractor 

(c)  CATEGORY  C 

I  Condition.  Chang*  in  end  article  die 
tates  requirement  for  new  itemisi  of  SE  which  super 
sedes  existing  itemts).  Superseded  item  si  may  be  mod 
ified  to  configuration  of  new  itemisi  retaining,  if 
possible,  the  capability  of  the  superseded  item 

2.  Action:  The  end  article  ECP  will  be  ac¬ 
companied  by  a  separate  ECP  for  each  item  of  SE  re¬ 
quiring  modification.  These  ECPs  shall  indicate  the 
modifications  necessary  to  make  the  SE  item  com¬ 
patible  with  the  new  end  item  configuration  and  will 
be  processed  concurrently  with  the  end  article  ECP  If 
the  ECP  to  the  end  article  establishes  a  requirement 
for  new  SE  a  new  SERD  will  be  processed, 
id)  CATEGORY  D 

/  Condition:  Changes  to  an  end  item  of 
SE  are  necessary  to  correct  a  design  deficiency  in  the 
support  equipment 

2.  Action:  Submit  SE  ECP  prepared  and 
submitted  in  accordance  with  MIL-STD  480.  accom¬ 
panied  by  a  SERD  reflecting  th*  new  item  If  mod¬ 
ification  is  required,  follow  procedures  for  category  C 
and  submit  a  separate  SERD  reflecting  the  general  de¬ 
scription  of  the  modification  material  required. 

(#)  CATEGORY  E: 

/  Condition:  Changes  to  an  end  item  of 
SE  are  necessary  to  implement  a  design  improvement 
All  items  will  not  necessarily  be  changed. 

2.  Action:  Submit  SE  ECP  prepared  and 
submitted  in  accordance  with  MIL  STD-480,  accoi-i 
parued  by  separate  SERDs  for  the  old  item  as  well  at 
the  new.  The  Government  will  determine  whether  all 
or  pm  of  the  delivered  items  will  be  modified  If  mod¬ 
ification  is  required,  follow  procedures  for  category  C 
and  submit  a  separate  SERD  reflecting  the  general  de¬ 
scription  of  the  modification  material  required 

'2)  Class  II  Charges  Class  II  changes  shall  be 
submitted  to  the  ACO  for  review  and  concurrence  in 
classification. 


5-9.  SE  Acquisition: 

a.  Initial  Order  The  initial  order  is  accompnsned 
b>  return  of  the  approved  SERD  and  issuance  of  an 
order  by  the  ordering  activity  The  quantity  of  SE 
items  to  be  acquireo  shall  be  in  accordance  with  Fig¬ 
ure  lb. 

b.  Follow-On  Orders  Based  on  a  review  of  the 
CSEL  or  SERD  by  the  Government,  the  contractor 
may  be  furnished  an  order  for  items  and  quantities  of 
SE  which  the  Government  determines  should  be  pro¬ 
cured  from  the  contractor.  The  contractor  will  proceed 
to  produce  or  procure  the  items  and  quantities  so 
ordered. 

c.  Release  Cancellation  Order- 

(1)  Release  In  the  event  the  contractor  has  not 
received  a  response  within  seventy-live  T5i  days  from 
date  of  the  initial  submission  of  a  SERD.  the  ordering 
activity  upon  request  from  the  contractor  and  subject 
to  availability  of  funds,  may  issue  an  order  for  the  SE 
itemisi  in  quantities  recommended  by  the  contractor 
on  the  SERD  or  as  otherwise  determined  by  the  order¬ 
ing  activity  The  contractor  wiil  notify  the  AA  and  the 
RA  in  writing  'electrical  transmission  preferred)  seven 
i  T i  days  pnor  to  the  effective  automatic  order  date. 

1 2)  Cancellation  Any  changes,  occasioned  by 
subsequent  directives  which  result  in  cancellation  ter¬ 
mination  charges  will  be  processed  and  settled  as  pro¬ 
vided  for  m  the  cancellation  termination  clause  of  the 
contract.  The  contractor,  in  effecting  such  order,  shall 
not  exceed  the  funds  authorized  for  the  SE  items. 
When  authorized  funds  are  insufficient,  the  contractor, 
shall  advise  the  ordering  activity 

d  SE  Selection  Reaeu  Meeting 

'll  Establishing  Requirement.  A  SE  selection  re¬ 
view  meeting  ma>  be  held  at  the  option  of  the  AA  RA 
when  it  is  determined  that  such  a  meeting  is  neces¬ 
sary  due  to  program  changes,  engineering  changes  to 
the  end  article  which  would  cause  significant  change 
in  items  and  quantities  of  5E  recommended  by  or  or¬ 
dered  from  the  contractor,  or  other  reasons  subject  to 
mutual  agreement  between  the  contractor  and 
Government. 

'2l  Contractor  Preparation  The  contractor  shall 
recommend  to  the  AA  a  date  and  place  for  holding  the 
SE  selection  review  meeting  The  contractor  must  en¬ 
sure  that  the  recommended  date  will  allow  sufficient 
time  for  ordering  of  the  SE  pnor  to  delivery  of  the  last 
end  article  The  contractor  shall  have  available  for  the 
meeting  all  technical  data  including  SERDs.  specifica¬ 
tions,  and  drawings  that  are  available  for  .he  CFE 
items  of  SE  listed  on  the  CSEL.  as  well  as  for  those 
items  for  which  requirements  developed  subsequent  to 
the  submission  of  the  last  CSEL  revision.  The  AA 
shall  specify  those  articles  desired  for  inspection  dur¬ 
ing  the  review  meeting.  The  contractor  will  advise  the 
AA  of  those  items  that  cannot  be  made  available  and 
-easons  therefore.  In  the  event  the  contract  does  not 
contain  a  specification  covering  drawings,  those  draw¬ 
ings  used  by  the  contractor  for  manufacturing  or  pro¬ 
curement  purposes  will  be  suitable  tor  review 
purposes 

■  .'Ji  The  meeting  The  government  team  shall  re¬ 
view  the  specifications  and  supporting  data  in  con¬ 
junction  with  the  CSEL  furnished  by  the  contractor 
The  team  shall  make  whatever  aaiustments  are  neces 
sary.  in  items  and  quantities  of  SE  previousiv  ordered 
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from  the  contractor  to  meet  current  program  require¬ 
ments  commensurate  witn  the  terms  of  the  contract. 

•  4 1  Conclusion.  The  contractor  shall,  not  later 
than  fifteen  >  15>  days  after  the  meeting,  in  accordance 
with  trie  quantities  and  destinations  specified  in  the 
DD  Form  l -123.  furnish  the  activities  revised  copies  of 
the  SERD'si  and  CSEL  reflecting  action  taken  Within 
fifteen  1 15>  days  after  receipt  of  the  revised  copies  of 
the  SEROisi  CSEL  or  as  negotiated  at  the  guidance 
conference,  the  procuring  activity  shall  provide  the 
contractor  written  authorization  to  proceed  and  or 
continue  with  the  design  fabrication  or  procurement  of 
the  items  and  quantities  of  the  SE  selected. 

f.  Changes  tv  Orders 

ill  Additions.  Unless  otherwise  negotiated,  the 
Government  may  at  any  time,  but  not  later  than  the 
scheduled  delivery  of  the  last  end  article  under  the 
contract,  order  additional  SE  Delivery  of  additional 
items  or  quantities  of  SE  included  in  the  order  or  re¬ 
vision  thereto,  shall  be  made  in  accordance  with  a 
schedule  agreed  upon  by  the  Government  and  the 
contractor. 

<2i  Deletions.  The  Government  may  at  any  time, 
in  writing,  delete  items  of  SE  on  order  Charges  with 
respect  to  any  items  and  quantities  of  SE  deleted  will 
be  processed  and  settled  as  nrovided  in  the  contract 
cancellation  terms 

5-10.  Delivery  of  Support  Equipment: 

a.  Time  of  Delivery  The  contractor  shall  propose 
delivery  schedules  in  accordance  with  DID  DI  P-6165 
within  thirty  i30>  days  after  receipt  of  an  order  from 
the  Government  Within  fifteen  il5>  days  after  receipt, 
the  Government  will  approve,  or  will  negotiate  to  re¬ 
vise.  the  contractor  s  proposed  delivery  on  a  specific 
SE  end  item  basis.  Failure  to  agree  on  the  delivery  of 
a  specific  SE  end  itemisi  will  not  affect  the  remaining 
items  when  the  Government  and  the  contractor  have 
reached  an  agreement.  The  approved  schedule  will  be 
incorporated  into  the  contract  at  the  time  of  incorpo¬ 
ration  of  the  Priced  SE  List.  The  contractor  shall  not 
deliver  SE  more  than  ninety  i90i  days  before  delivery 
of  the  training  equipment  first  end  article,  or  the  date 
of  scheduled  delivery,  uniess  SE  delivery  is  specified 
or  approved  by  the  Government.  The  early  delivery  of 
these  items  of  SE  normally  will  be  limited  to  ship/ 
base  installed  items. 

b.  Point  of  Delivery  The  ordering  activity  shall  no¬ 
tify  the  contractor,  in  writing,  as  to  the  destination  of 
SE  to  be  furnished  hereunder  at  least  120  days  in  ad¬ 
vance  of  the  scheduled  delivery,  or  on  a  date  mutually 
agreed  upon  by  the  contractor  and  the  ordering  activ- 
ity  When  believed  to  be  tn  the  best  interest  of  the  Gov- 
eminent,  the  contractor  shall  recommend  to  the  order¬ 
ing  activity  a  schedule  for  shipment  of  SE  on  order 
with  the  contractor  but  manufactured  in  plants  other 
than  the  contractor's  plant  or  plants.  When  approved 
and  directed  by  the  ordering  activity,  the  contractor 
shall  instruct  the  manufacturer  to  make  direct  ship¬ 
ment  to  the  designated  destination  The  contractor 
shall  not  deliver  any  SE  unless  the  shipping  papers, 
including  the  packing  sheets,  fully  identify  each  item 
to  be  shipped  in  accordance  with  the  identification  es¬ 
tablished  for  such  items  except  that  the  contractor 
may  shorten  the  description  by  listing  the  principal 
noun  and  type  designators  in  addition  to  one  or  two 
descriptive  modifiers  Unless  otherwise  authorized  by 
the  SE  ordering  activity,  the  contractor  shall  not  ship 


equipment  for  operational  or  training  purposes  with¬ 
out  an  NSN  which  snail  be  furnished  by  the  Govern¬ 
ment  pn^r  'o  required  delivery. 

c.  Delinquency  Delivery  Report  The  conn-actor 
shall  furnish  delinquency  delivery  reports  for  SE  in 
accordance  with  DID  DI  P-6165. 

d.  Preservation.  Paceaging.  Packing,  and  Marking 
Requirements.  Preservation,  packaging,  packing  and 
marking  applicable  to  SE  shall  be  provided  in  accord¬ 
ance  with  the  contract  requirements  ichapter  3i. 

5-11.  SE  For  Training.The  procedures,  terms,  condi¬ 
tions.  and  delivery  of  SE  selected  for  use  as  training 
equipment  or  to  support  training  equipment  shall  be 
in  consonance  with  the  applicable  training  need  date 
cited  in  the  contract.  To  assure  timely  delivery  of  SE 
for  training  equipment,  the  contractor  shall  exercise 
special  surveillance  over  the  procedures,  terms,  condi¬ 
tions.  and  deliverv  of  such  SE  as  outlined  in  this  docu¬ 
ment.  Such  special  surveillance  requires  tnat  the  con¬ 
tractor  clearly  identify  on  the  initial  identification 
document  iSERD)  those  SE  items  required  for  training 
isee  chapter  13). 

5-12.  Pricing: 

a.  Priced  Support  Equipment  Lists  ' PSELsi  and 
Revisions  Thereto  The  PSE/Ls  reflecting  the  material 
and  effort  covered  by  the  Government  orders  shall  be 
submitted  to  the  ACO  by  the  contractor  within  60 
days  after  receipt  of  the  order  or  as  negotiated  be¬ 
tween  the  procuring  activity  and  the  contractor  A  sep¬ 
arate  pnced  list  shall  be  submitted  by  the  contractor 
for  each  item  or  subitem  under  this  contract. 

b  SE  Exhibits  If  the  item  prices  and  quantities 
contained  in  the  PSEL  and  in  the  cancellation  adden¬ 
dum.  U  any.  are  acceptable  to  the  Government,  the  list 
shall  be  incorporated  by  supplemental  agreement  into 
the  contract  as  the  SE  exhibit.  If  the  PSEL  is  not  ac¬ 
ceptable  as  submitted,  prices  shall  be  established  by 
agreement.  Failure  to  agree  upon  items,  quantioes.  or 
prices  to  be  included  in  the  SE  exhibit  shall  be  a  dis¬ 
pute  concerning  a  question  of  fact  within  the  meaning 
of  the  clause  of  the  contract  entitled  Disputes;''  how¬ 
ever  such  dispute  shall  be  limited  to  the  affected  items 
and  shall  not  delay  the  establishment  of  the  remain¬ 
der  of  the  list  as  the  SE  exhibit. 

c  Revision  of  Exhibit  Each  60  days,  as  necessary, 
beginning  with  the  establishment  of  the  SE  exhibit 
and  continuing  until  completion  of  deliveries  of  all 
items  to  be  delivered  under  the  contract,  the  contractor 
shall  prepare  and  submit  to  the  ACO  a  proposal  for  re¬ 
vising  the  SE  exhibit  to  date  covering  effort,  new 
items,  or  increased  or  decreased  quantities  of  items 
Each  such  proposal  shall  conform  to  the  format  of 
DID  DI-V-6184  to  include  proposed  price  and  indicate 
all  changes  and  revisions  approved  by  the  Govern¬ 
ment  Upon  the  establishment  of  firm  prices  in  accord¬ 
ance  with  the  provisions  of  b  above,  the  revised  SE  ex¬ 
hibit  will  be  incorporated  by  supplemental  agreement 
to  the  contract. 

5-13.  Funding.  The  contractor  shall  submit  report  of 
funding  requirements  and  status  in  accordance  with 
DID  DI-F-6127  The  report  shall  be  submitted  via  the 
ACO  to  the  destination  specified  on  the  DD  Form 
1423  The  report  shall  cover  each  SE  contract  line 
item  as  well  as  each  different  funding  citation,  in  the 
end  article  contract  The  report  snail  be  submitted  to 
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the  ACO  by  the  tenth  of  each  month  and  shall  reflect 
a  cut-off  date  of  funding  status  as  of  the  last  working 
day  of  the  previous  month. 
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APPENDIX  D 


SUMMARY  OF  JOINT  SERVICE  GUIDELINES 
FOR  AUTOMATIC  TEST  EQUIPMENT  ACQUISITION 

This  appendix  summarizes  two  Joint  Service  guides  prepared  under  the  auspices  of  the  Joint 
Logistics  Commanders  (JLC)  Panel  on  Automatic  Testing  Those  guides  are: 

•  "Joint  Service  Automatic  Testing  (AT)  Acquisition  Planning  Guide,”  19  March  1981 

•  "Joint  Service  Weapon  System  Acquisition  Review  Guidelines  for  Automatic  Testing 
(AT),”  19  March  1981. 

Both  guides  focus  on  automatic  testing,  including  design  for  testability,  built-in  test,  off-line  auto¬ 
matic  test  equipment  (ATE),  and  test  program  sets  (TPSs). 

The  planning  guide  provides  an  overview  of  the  acquisition  activities  as  they  relate  to  auto¬ 
matic  testing  (see  Figure  D-l).  It  stresses  the  application  of  logistic  support  analysis,  in  accordance 
with  Military  Standard  1388,  as  the  most  important  acquisition  activity  for  the  development  and 
implementation  of  the  automatic  testing  support  system;  it  points  out  the  preference  for  general- 
purpose  ATE  over  system-peculiar  ATE  to  restrict  the  variety  of  ATE  in  the  inventory;  it  encourages 
standardization  of  ATE  hardware  and  software  within  and  among  the  Military  Services;  and  it 
suggests  that  environmental  specifications  be  relaxed,  when  practical,  to  permit  use  of  commercially 
available  ATE  but  warns  about  the  need  to  ensure  logistic  support  of  commercial  equipment.  The 
planning  guide  describes  the  major  activities  in  each  phase  of  the  weapons  system  acquisition 
program.  In  the  concept  exploration  phase,  tradeoff  analysis  and  selection  of  a  cost-effective  ATE 
scenario  are  identified  as  the  most  critical  activities.  It  suggests  that  models  such  as  the  Support 
Equipment  Selection  Analysis  (described  in  Appendix  E)  be  used  in  ATE  scenario  development.  In 
the  demonstration  and  validation  phase,  logistic  support  analysis  and  determination  of  automatic 
testing  requirements  are  the  central  activities.  Automatic  testing  requirements  include  design  for 
testability  and  built-in  test;  with  respect  to  ATE,  the  focus  is  on  unit-under-test/built-in-test/ATE 


D-l 


compatibility  and  further  refinement  of  the  ATE  scenario  analysis  conducted  earlier.  The  major 
activities  in  the  full-scale  engineering  development  phase  are  refinement  of  the  logistic  support 
analysis  and  level  of  repair  analysis  for  each  configuration  item.  Once  these  activities  have  been 
accomplished,  the  test  requirements  analysis  can  be  completed  and  test  requirements  documents  can 
be  prepared  for  each  unit-under-test.  At  that  time,  final  ATE  selection  or  development  specifications 
should  be  completed  so  that  development  of  test  program  sets  can  be  initiated.  This  phase  is  to  be 
completed  with  various  test  and  evaluation  efforts  and  demonstrations.  The  guide  continues  by 
describing  various  activities  in  the  production  phase  with  emphasis  on  configuration  management 
requirements  for  TPSs. 

The  second  guide  lists  some  of  the  key  questions  to  be  considered  at  appropriate  review  points 
in  the  weapons  system  acquisition  process.  Those  questions  are  illustrated  in  Table  D-l. 


IGURKD1  FUNCTIONAL  FLOW  OF  ACQUISITION  ACTIVITIES 
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APPENDIX  E 


SUMMARY  OF  SUPPORT  EQUIPMENT  SELECTION  ANALYSIS  GUIDE  (NAVY) 

This  appendix  summarizes  Naval  Material  Command  Publication  9407,  Support  Equipment 
Selection  Analysis  Guide.  November  1981.  This  guide  describes  the  Navy’s  support  equipment  selec 
tion  process  and  associated  analytical  methodologies  for  evaluating  automatic  test  equipment  (ATE) 
alternatives  for  meeting  given  test  requirements.  The  analysis  is  intended  to  be  conducted  during  the 
demonstration  and  validation  phase  of  a  weapons  system  acquisition  program  and  to  be  completed 
early  in  the  full-scale  engineering  development  phase  with  identification  of  the  optimum  ATE  The 
analysis  is  normally  tailored  to  the  specific  circumstances  (considering  need,  schedule,  and  funds 
available)  and  is  either  conducted  in-house,  such  as  by  the  Naval  Air  Engineering  Center,  or  con¬ 
tracted  out  to  a  third  party  !  it  is  essentially  a  Government  tool  for  identifying  the  best  ATE, 
considering  current  ATE  inventory  and  Navy  policies  and  programs,  in  lieu  of  the  support  equipment 
recommendation  data  prepared  by  prime  contractors.  The  latter  approach  invariably  results  in 
system-peculiar  ATE  and  schedule  slippage,  both  of  which  can  be  avoided  through  the  disciplined 
application  of  support  equipment  selection  analysis  (SESA).  The  SESA  comprises  the  following 
six  consecutive  analyses  conducted  for  each  ATE  alternative:  (1)  test  requirements,  (2)  ATE  capabil¬ 
ities,  (3)  unit-under-test  (UUTVATE  compatibility,  (4)  ATE  loading,  (5)  life  cycle  cost,  and  (6)  risk 
assessment.  The  analytic  methods  are  briefly  described  below. 

ELECTRONICS  TEST  REQUIREMENTS  ANALYSIS 

The  first  SESA  consists  of  collecting  CUT  test  requirements,  usually  from  a  preliminary  test 
requirement  document  (TRD)  at  the  weapon  replaceable  assembly  level.  Shop  replaceable  assembly 


'For  example,  a  contractor  application  of  the  analysis  is  described  in:  Dr  H  D.  Kimp  (M&T 
Company,  King  of  Prussia,  Pennsylvania),  Support  Equipment  Selection  Analysis  (SESA)  for  the 
Navy  Standard  Airborne  Computer  Set  (AN/AYK-141 V).  Report  No.  NAEC-92-138  (Lakehurst, 
New  Jersey:  Naval  Air  Engineering  Center,  17  November  1980) 


data  are  seldom  available  until  start  of  full-scale  engineering  development.  [Naval  Air  Systems 
Command  uses  its  own  specification  for  TRDs,  Military  Standard  (MIL-STD)  -2076(AS),  "Unit  Under 
Test  Compatibility  with  Automatic  Test  Equipment,  General  Requirements  For”;  1  March  1978, 
which  is  more  detailed  than  the  other  available  specifications.2]  If  the  Contract  Data  Requirements 
List  does  not  include  delivery  of  a  preliminary  TRD,  then  that  data  must  be  collected  by  the  SESA 
team. 

As  specified  by  MIL-STD-2076,  the  TRD  includes  both  UUT  data  and  test  requirements  data  in 
a  prescribed  format.  The  UUT  data  include  drawings,  schematics,  wiring  diagrams,  logic  diagrams, 
functional  block  diagrams,  parts  list,  design  data,  failure  data,  performance  characteristics,  and 
interface  descriptions.  (Some  or  many  of  these  data  may  not  be  available  in  a  preliminary  TRD  )  The 
UUT  test  requirements  are  specified  both  in  parametric  form  and  in  ATLAS  (Abbreviated  Test 
Language  for  All  Systems).  (The  latter  is,  of  course,  not  available  in  a  preliminary  TRD.)  Both  the 
UUT  data  and  test  requirements  are  then  summarized  on  UUT  data  sheets  grouped  into  four  func¬ 
tional  areas,  power  supplies,  digital  signals,  analog  signals,  and  servo/synchro  signals. 

SUPPORT  EQUIPMENT  CAPABILITIES  ANALYSIS 

The  second  SESA  is  a  systematic  effort  to  delineate  the  test  capabilities  of  each  candidate  ATE. 
Like  the  first,  it  is  a  data  collection  effort  but  includes  a  technical  evaluation  of  the  adequacy  of  test 
capabilities.  The  sources  of  data  include  the  ATE  data  banks  maintained  by  Navy  activities  (such  as 
the  Naval  Air  Engineering  Center  and  Fleet  Analysis  Center)  and  other  Department  of  Defense 
Components  (e  g.,  San  Antonio  Air  Logistics  Center,  San  Antonio,  Texas)  as  well  as  ATE  manu¬ 
facturers.  The  data  are  then  summarized  on  support  equipment  data  sheets  containing  general 
information  and  test  capability  specifications  in  the  same  four  functional  areas  as  listed  for  UUT  data 
sheets. 

2MIL-STD-1519  (USAF),  "Preparation  of  Test  Requirements  Document,”  17  September  1971 
(with  Notice  1,  1  August  1977).  MIL-STD-1345B  (Navy),  "Test  Requirements  Document,” 
10  February  1981  The  Army’s  standard  is  a  contractor-prepared  document,  Design  Standard  for 
the  Development  of  AN/USM-410  Test  Program  Sets.  RCA  Document  CR82-588-04,  December  1982 
(revised  February  1984).  This  document  does  not  include  the  term  TRD,  but  directs  that  test 
specifications  be  included  in  the  UUT  data  package,  when  possible 


COMPATIBILITY  ANALYSIS 

The  third  SESA  consists  of  a  detailed  comparison  of  UUT  test  requirements  with  ATE  capabil¬ 
ities  to  determine  tester  adequacy,  identify  any  deficiencies  and  possible  corrective  actions,  estimate 
cost  factors,  evaluate  logistics  support  requirements,  and  assess  technical  risks.  Depending  on  the 
scope  of  the  SESA  effort  and  the  number  of  available  ATE  candidates,  this  analysis  may  also  include 
technical  definition  of  new  ATE  alternatives. 

For  the  ATE-UUT  comparison,  available  analytic  and  simulation  models  may  be  used.  For 
each  mismatch  of  test  requirements  and  capabilities,  further  technical  analysis  is  required  to  exam¬ 
ine  potential  solutions,  such  as  new  interface  devices,  software  modifications,  or  ATE  modifications  or 
as  a  last  resort,  new  ATE.  The  result  is  a  list  of  ATE  alternatives  for  each  UUT.  The  engineering  and 
unit  production  costs  of  these  alternatives  are  then  estimated.  Both  the  engineering  and  unit 
production  costs  are  to  include  only  those  costs  associated  with  supporting  the  specific  UUT  under 
consideration.  To  illustrate,  for  an  existing  tester  the  costs  would  include  those  for  the  interface 
device,  test  program  set,  and  any  tester  modifications.  Once  those  costs  are  estimated,  the  logistics 
support  costs  are  estimated  for  each  ATE  alternative  as  well  as  the  associated  technical  risk. 

ATE  LOADING  ANALYSIS 

The  fourth  SESA  consists  of  evaluating  the  workload  for  each  ATE  alternative  caused  by  the 
supported  weapons  system  in  order  to  determine  the  number  of  testers  required  as  well  as  their 
utilization.  This  analysis  requires  an  ATE  installation  site  survey,  information  on  weapons  system 
mission  requirements  and  reliability  characteristics,  estimated  test  run  times,  tester  capacity  in 
terms  of  available  tester  hours  per  month  as  a  function  of  shift  operations,  ATE  reliability  and 
maintainability  characteristics,  and  other  existing  workload  (other  than  the  target  UUTs).  The 
guide  provides  several  factors  for  converting  theoretical  mean  time  between  failure  values  to  mean 
time  between  maintenance  values  so  as  to  account  for  false  removals.  The  guide  also  suggests  that 
the  user  account  for  inefficiencies  (ATE  operator  errors,  job  diversions)  by  reducing  tester  available 
hours  by  an  empirically  determined  factor  of  30  percent.  The  product  of  this  analysis  is  a  matrix 
showing  the  number  of  testers  requested  at  each  site  and  their  utilization  for  each  ATE  alternative 


LIFE  CYCLE  COST  ANALYSIS 

The  fifth  SESA  consists  of  quantifying  the  costs  associated  with  the  best  ATE  alternatives. 
This  analysis  is  not  a  complete  life-cycle-costing  effort,  but  it  is  limited  to  the  marginal  costs  that 
would  be  incurred  for  existing  ATE.  Those  costs  include  nonrecurring,  recurring,  and  operating/ 
support  costs.  The  guide  includes  a  detailed  cost  element  breakdown  structure  to  aid  in  making  the 
cost  estimating  effort  consistent.  It  also  lists  various  sources  for  cost  data. 

PROGRAM  AND  RISK  EVALUATION 

The  final  SESA,  conducted  concurrently  with  the  previous  analysis,  addresses  the  risks 
involved  for  each  ATE  alternative.  The  potential  risks  include  both  technical  risks  (ATE  and/or  test 
program  sets)  and  management  risks.  The  guide  devotes  32  pages  to  a  discussion  of  these  risks,  but 
acknowledges  that  quantifying  these  risks  in  terms  of  probability  of  occurrence  and  resulting  cost  and 


schedule  impacts  may  be  an  elusive  task. 


APPENDIX  F 


NAVY’S  PLANNED  APPROACH  TO  SERVICE  WARRANTIES  FOR  COMMON  ETE 

(UNOFFICIAL) 

This  appendix  provides  the  draft  contract  clause  on  "extended  warranty”  under  the  Navy’s  pro¬ 
posed  approach  for  procurement  of  common  electronic  test  equipment  (ETE),  whereby  the  service  cost 
is  included  in  the  procurement  cost.  The  following  pages  have  been  retyped  from  a  copy  of  the  draft 
contract  clause  received  from  the  Navy. 

EXTENDED  WARRANTY 

The  Contractor  warrants  that  the  equipment  furnished  under  this  contract  shall  be  free  from 
defects  in  materials  and  workmanship  for  a  period  of  5  years  and  shall  provide  an  option  wherein  the 
Navy  may  negotiate  an  extension  of  the  warranted  period  for  an  additional  5  years.  If  products  are 
found  defective  within  the  warranty  period,  the  Contractor  shall  repair  the  defective  equipment 
without  charge  for  parts  and  labor.  This  warranty  shall  not  preclude  emergency  repair  and  calibra¬ 
tion  actions  by  the  Navy.  Such  actions  shall  be  fully  documented  to  the  Contractor  and  subject  to  the 
"abuse”  clause  contained  herein. 

The  Contractor  shall  designate  at  least  ten  service  centers  within  the  contiguous  United 
States,  and  six  service  centers  outside  the  contiguous  United  States  for  the  purpose  of  equipment 
receipt  from,  and  return  to  the  Navy.  The  domestic  service  centers  shall  be  located  in  or  near 
Portsmouth,  New  Hampshire;  Philadelphia,  Pennsylvania.  Norfolk,  Virginia;  Charleston,  South 
Carolina;  Orlando,  Florida,  San  Diego,  California;  Los  Angeles,  California;  San  Francisco,  California; 
Chicago,  Illinois;  and  Seattle,  Washington  Overseas  service  centers  shall  be  located  in  or  near 
London  (England),  Naples  (Italy),  Athens  (Greece),  Tokyo  (Japan),  Manila  (The  Philippines),  and 
Honolulu,  Hawaii.  The  Contractor  shall  mark  each  equipment  delivered  under  this  warranty  with  a 
large  decal  or  other  method  to  make  it  readily  identifiable  as  a  warranty  item.  The  warranty  decal 


shall  identify  an  800  telephone  number  provided  by  the  Contractor  that  can  be  used  by  the  Navy  to 
arrange  calibration  and  repair  actions  into  and  from  the  designated  service  centers. 

Under  the  terms  of  this  warranty,  operable  equipment  will  be  delivered  to  a  service  center  by 
the  Navy  on  a  scheduled  annual  basis  for  refurbishment,  calibration  (including  incidental  repairs), 
and  affixed  with  a  calibration  sticker.  Inoperable  equipment  delivered  to  a  service  center  shall  be 
repaired  and  calibration  recertified  for  a  12-month  period.  Replaced  parts  shall  become  the  property 
of  the  Contractor.  For  each  calibration  and  repair  action,  the  Contractor  shall  annotate  a  METER 
card  (calibration  label)  provided  with  the  equipment  in  accordance  with  the  MEASURE1  Users 
Manual.  If  the  Contractor  deduces  that  the  inoperable  equipment  has  been  abused,  he  shall  notify 
the  user  and  arrange  for  nonwarranty  repair.  Commander,  Space  and  Naval  Warfare  Systems  Com¬ 
mand,  Code  84110,  Washington,  D  C.  20363-5100  shall  be  provided  written  details  of  the  equipment 
abuse.  Equipment  shall  be  returned  to  the  service  center  within  two  calendar  weeks  for  locations 
within  the  contiguous  United  States,  and  within  three  calendar  weeks  when  shipping  is  required  out¬ 
side  the  contiguous  United  States  (abused  equipment  shall  be  exempted  from  turnaround-time  con¬ 
straints).  All  shipping  charges,  taxes,  duties,  and  associated  costs  incurred  from  the  time  the  Navy 
delivers  an  equipment  to  a  service  center,  to  the  time  the  Navy  picks  up  the  equipment,  shall  be  borne 
by  the  Contractor. 

A  written,  semiannual  report  shall  be  made  to  Space  and  Naval  Warfare  Weapons  Command 
Code  841 10  by  the  Contractor.  The  report  shall  delineate  repair  and  calibration  transactions  with 
dates  of  equipment  receipt  at  a  service  center  and  when  the  equipment  is  available  for  pick-up  by  the 
Navy.  Transactions  shall  be  annotated  with  a  positive  value  in  days  ahead  of  its  respective  delivery 
schedule  and  a  negative  value  in  days  behind.  A  zero  vaiue  shall  indicate  on  schedule.  The  values 
shall  be  totaled  for  the  semiannual  period.  The  Contractor  shall  incur  a  penalty  at  the  equivalent 
rate  of  $200  per  day  for  a  negative  total.  This  penalized  amount  will  be  subsequently  used  by  the 


'Metrology  Automated  System  for  Uniform  Recall  and  Reporting 
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Navy  to  offset  costs  for  scheduled  Contractor’s  services  for  repair  of  abused  equipment  or  other 
nonwarranted  repair  services  at  the  current  published  Contractor  charges. 

For  the  purpose  of  this  extended  warranty,  the  following  definitions  apply: 

•  Calibraf  ion:  The  process  of  comparing  an  instrument  of  unverified  accuracy  to  a  certified 
standard  of  greater  accuracy,  traceable  to  the  National  Bureau  of  Standards,  to  detect  and 
correct  all  variations  from  the  manufacturer’s  published  specifications  for  the  instrument. 

•  Incidental  repair:  Those  repairs  found  necessary  during  calibration  of  operable  equipment 
to  bring  it  within  its  specified  tolerances,  including  the  replacement  of  parts  which  have 
changed  value  sufficiently  to  prevent  certification  but  do  not  otherwise  render  the  equip¬ 
ment  inoperative.  The  term  "incidental  repair”  does  not  apply  to  inoperable  equipment. 

•  Operable  equipment:  Equipment  which,  from  its  most  recent  performance  history  and  a 
cursory  electrical  and  physical  examination,  displays  an  indication  of  operational  perform¬ 
ance  for  all  required  functions  when  submitted  for  calibration. 

•  Repair:  The  restoring  of  the  instrument,  both  electrically  and  mechanically,  so  that  it  will 
meet  all  of  the  instrument  manufacturer’s  published  specifications  as  verified  by  cali¬ 
bration. 

•  Refurbishment:  Preventative  maintenance  actions  intended  to  maintain  the  equipment  in 
ready-for-use  status. 

•  Abused  equipment: 

Physical  abuse:  Damage  associated  with  the  improper  handling  of  test  equipment  such 
as  dropping,  throwing,  or  other  mechanical  stress  which  results  in  damage  not  asso¬ 
ciated  with  normal  use. 

Electrical  abuse.  Damage  to  circuitry  because  of  overloading  (such  as  the  input  of 
excessive  RF  energy). 

Cannibalization:  Defined  as  the  stripping  of  good  components  or  assemblies  from  one 
instrument  to  repair  another  instrument.  This  abuse  is  indicated  by  the  presence  of 
two  or  more  unrelated  failures 

Improper  or  detrimental  emergency  repair  by  the  Navy. 

•  Service  Center:  A  designated  point  of  entry  for  the  receipt  of  equipment  from  the  Navy  and 
return  thereto. 


_ UNCLASSIFIED _ 

SECURITY  CLASSIFICATION  OF  THIS  PAGE 


la.  REPORT  SECURITY  CLASSIFICATION 
Unclassified 


2a  SECURITY  CLASSIFICATION  AUTHORITY 


REPORT  DOCUMENTATION  PAGE 


lb  RESTRICTIVE  MARKINGS 


2b. DECLASSIFICATION  /DOWNGRADING  SCHEDULE 


3  DISTRIBUTION  /  AVAILABILITY  OF  REPORT 

"A"  Approved  for  Public  Release;  distribution  unlimited. 


4  PERFORMING  ORGANIZATION  REPORT  NUM8ER(S) 
LM1  Task  ML504 


5  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a  NAME  OF  PERFORMING  ORGANIZATION  6b  OFFICE  SYMBOL  7a.  NAME  OF  MONITORING  ORGANIZATION 

Logistics  Management  Institute  ^  aPP,lca^le) 


6c  ADDRESS  (City.  State,  and  ZIP  Code ) 

6400  Goldsboro  Road 
Bethesda,  Maryland  20817-5886 


7b.  ADDRESS  (City,  State,  and  ZIP  Code) 


8a  NAME  OF  FUNDING /SPONSORING 
ORGANIZATION 
ASD<  A&Li 


8c  ADDRESS  (City,  State,  and  ZIP  Code) 


8b  OFFICE  SYMBOL 
(if  applicable) 


9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
MDA903-85-C-0139 


10.  SOURCE  OF  FUNDING  NUMBERS 

PROGRAM 
ELEMENT  NO 

PROJECT 

NO 

TASK 

NO 

WORK  UNIT 
ACCESSION  NO 

1 1  TITLE  (Include  Security  Classification) 

Management  of  Electronic  Test  Equipment 
Volume  III:  Organizations,  Policies,  and  Procedures 


12  PERSONAL  AUTHOR(S) 
Frans  Nauta 


13a  TYPE  OF  REPORT 
Final 


16  SUPPLEMENTARY  NOTATION 


13b  TIME  COVERED 
FROM  TO 


15  PAGE  COUNT 
185 


17 

COSATI  CODES 

FIELD 

GROUP 

SUB-GROUP 

1 8  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

Test  Equipment,  Electronic  Test  Equipment,  Automatic  Test  Equipment.  Test  Program  Sets, 
Management  and  Support  of  Test  Equipment 


19  ABSTRACT  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

Since  the  early  1970’s,  the  Office  of  the  Secretary  of  Defense,  the  Military  Departments,  and  industry  have  sponsored  numerous  studies  of  test 
equipment,  formed  several  joint  panels  to  investigate  selected  technical  issues,  and  initiated  a  variety  of  programs  to  correct  identified  problems.  Despite 
such  attention,  the  Department  of  Defense  still  faces  many  significant  problems  with  fielded  test  equipment. 

In  a  previous  report.  Test  Equipment  Management.  January  1985,  we  summarized  the  nature  and  extent  of  those  problems  and  recommended  the 
Assistant  Secretary  of  Defense  (Manpower  Installations,  and  Logistics),  ASDlMI&L),  take  the  lead  in  effecting  needed  improvements  in  test  equipment 
management  and  support.  The  ASD(  MI&Li  concurred  with  that  recommendation  and  established,  via  an  action  memorandum  for  the  Under  Secretaries  of 
the  Military  Departments,  dated  26  June  1985,  a  ”DoD  Test  Equipment  Management  Improvement  Program'1  under  the  overall  guidance  of  his  Mainte¬ 
nance  Directorate. 

Since  the  issuance  of  that  action  memorandum,  the  Maintenance  Directorate  has  been  coordinating  DoD wide  efforts  to  implement  the  DoD  Test 
Equipment  Management  Improvement  Program.  This  report,  which  is  published  in  four  volumes,  bolsters  ihe  Maintenance  Directorate's  initiative 
Volume  I  lays  out  a  specific  program  of  action  for  the  Assistant  Secretary  of  Defense  (Acquisition  and  Logistics!  to  serve  as  a  cornerstone  for  the  DoD  Test 
Equipment  Management  Improvement  Program;  Volume  II  reviews  previous  studies  and  initiatives  pertaining  to  test  equipment  management  and 
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